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ABSTRACT \ ^ ' ■ 

Progress made on the development of a plan for 
evaluating the cost performance and educational benefits of the 
TICCIT and PLATO projects iis Summarized in this anniial report. The 
basic focus of the first section of the report. Cost Analysis, is tp 
provide answers to the fundjamental question of how much these two CAI 
methods cost compared with jsoine measure of non-CAI ^baseline^* 
educational costs* The second section. Educational Analysis, 
describes^ the l)a€feline educitiorxal analysis that will be implemented 
during 1972-73. Attention is^ given to achievement and attitude 
m^asfurement, the experimental design, and plans for data collection. 
In the third section. Technical Analysis, the progress of the two 
systems to-date is summarizedV^as based on site visits to both the 
University of Illinois ^nd MITOE, in relation to system development, . 
decisions that will be important in the determination of the nature 
of the, final system, apd the adequacy of the method for collecting 
system performance data. An extensive bibliography, with brief 
annptaitions, lists recent writings related tio the 'subject of 
computer^-baseH education in general and those technically related to 
the research, development, arid evaluation concerns of the project. 
The two appendices to the report^ are: A. Survey qf Instructor 
Activities ahd Attitudes, «and 6* Attitudes toward Reading 
Questionnaires — Grade 2 and Grades ^ and.^6. (For related documents, 
see TM 002 297, 298, and 386.) (DB) 
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ANNUAL REPtDRT ' • ' 

Period: 7 Febraary, 1972 to 3.1 Juiy, 1972 ( 

t 

The following .report is submitted in partial response to Contract 

NSF Q-IZ]^ The major objective of^the report is to summarize progress 

on the development of a plan- for evaluating the cost performance and 

educational benefits of the TICCIT^and PLATO Droiects. . , 

* .J > 

'Detailed plans for the baseline cost. and educational data collec- 
tions are presented as well as our current thoughts on th^ design: 
of the technical evaluation . In addit ian , we prpvide an annotated 
bibliography of recent publications relating to the use of computers 
^in the instructional process. " ' 

We are satisfied that progress in^the implementation o*f PLATO 
•and TICeiT has been reasonable , with the exception of the course 

develapment area. Some delays have been enp^^untered in the develop.- . 

ment of the delivery systems; however, forCthe most* part,. these delays 
• have led to more carefully defined plar?s .fo*r the two systems and 

to greater promise of a useful demonstration. ' ^ 
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a: 



^ P^erhaps the most uncertain and potentially disturbing aspect ^ 
of/the development of both the MITRE and the University of Illinois 
projects has to do with the authoring process. No matter how well 
designed and reliable the hardware, or how flexible and powerful the* 
supp-orting software, the educational' progress of students using each 
system will be limited by the nat'lire. of the curricular' materials 
presented to them. In quite opposite ways, both- projects^run the 
distinct risk of unde res tlmat ing . what is required to produce an 
effective curriculum. MITRE ' s* approach to curriculum design could 

• easily, err in the direction of excessive and premature systematiza- 
txbn of materials lacking in adequ^e Realization of t,he potential- 
ities of the medium tor delivery. . Illinois, on the other hand, . ♦ ' 
could ea^gily^ err in the direction ot ^excessive ' rel iance apon- highly 
inspired adjunctive exercises demdnd t-r ating the great versatility 

)and creative potential of the medium but inadequately supported by 
comprehensive, systematic coverage of « cur rlcular objectives. In 
our vtew, these risks are very real and must be recognized and dealt 
with if the delivery systems are to be given a fair trial and evalu- 
ation . ^ * ^ , j^t — - 

* ^ .... 
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To emphasi^ie thJ.8 point by analogy, suppose'that the Air Force '\ 
bad under development, a radically new design of fighter bombers' and 
was preparing to test their effectiveness under combat "conditions • 
We would expect them to insist' upon experienced pilots and highly 
trained flight crews* And we would not regard it as a fair test 
of the new aircraft if the crews were ^selecte^ on the basis of 
interest, ground artillery experience, knowledge of the combat 
terrain, or any other tangentidl criterion. Analggies are sometTlmes / 
misleading, and we do not wish to push this one too far. The count- 
erpart of an experienced flight crew, thoroughly skilled for coping 
with both a new craft and with combat conditions, W9uld be extremely 
hprd to find or assemble in the field of CAI. .MI'TRE and Illinpis \ 
are both to be commended for the ingenuity and perseverance with 
which they have faced the' task of authoring for new media .and for 
mi^ec±£±c curricular problem areas, but the fact remains that both' 
may have discounted* to*o much the skilX> training, and experience 
needed for effective authoring, 

' V- • 

Illinois seems to have been carried away by their own enthusi- 
asm tor the power of the. new medium, arid to. have given insuf f ici-ent 
attentionvto the detailed work of defining educational objectives 
and implementing scripts. The possibility that the content of a 
PLATO program could be based on an inadequate ' educat ional strategy 
hasT riot, been- faced. MITRE, on the other hand, has'placed inordinate, 
faith in educational o^bjectives and adopted an asarembly-line author- 
ing p^rocess before even a small-scale protot-ype of a furrfioning 
program could be demons^t rated or revised. Contrary to the experi- 
ence of others', they Sre attempting the" basic authoring input into 
the assembly line witm^ut the benefit of early direct contact with 
an operating system. A very difficult tasjc is left to the instruc- 
tional 'designer to take the bits and . p ieces' and assemble . them into 
an imagin£(ti've, challenging lesson. 

Some risks are in^evitable for any large, ground-breaking , innova- 
tion, and both- the MITRE and Univer!sity of Illinois projects would ^ 
be impossible without accepting risks. It is, nevertheless, import- 

»tant to reduce the sources of risk in every reasonable way. One 
can perhaps justify MITRE's insistence upon assembly-like auth-orlng 

'itfechniques as essential to the eventual economic viabiliCy of their 
overall system, or Illinois' oonfid^nce. in inexperienced authors as 
part of a grand conception of how an essentially new medium must 
develpp its own freedom and style "independent of the conventions 
and discipline of textbook authoring. But it should then be recognized 
that any evaluation of these * systems And of students' educational 
progress will be to some degree jeopardized. In any case, . 
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unnecessary aspects of the compounded risk should "be eliminated — 
MITRE by exposing authors to the medium and its possibilities as 
feoon as possible, and Illinois by taking steps to assure that basic 
educational objectives are adequately covered. 



/ 



COST ANALYSIS 



This section describes the structure of a cost-collection system 
thafi has been designed to provide much of the basic data, to be used ^ 
b^ th-e cost-analysis team for its financial analyses. Also included 
is a brief review of the cost team's activity through June 30, 1972, 
as^ well as a calendar of plans and objectives fof implementing and 
reviewing the proposed cost-collection program for fiscal 1972-1973. 

¥he principal purpose of capturing both direct and indirect costs 
over time is to explain frpm a finanqial standpoint the development, 
implementation, and operation of each system and the System's rela- 
tionship to the educational process. The ba'^ic focus of the' cTost^ 
•analyses is to provide, as relial>ly and objectively as posslblja, 
answers to the fundamental but difficult question of how mu these 
two CAI methods cost compared to • ^ome measure of non-CAI "baseline" 
educational costs. Since both of the 'CAI systems are experimental, 
we feel it imperativ^e to extrapolate trends from the. collected cost . 
components in order to make reasonable estimates of future costs of 
these or similar CAI systems. It is our conclusion that the compTex 
financial questions that are bound to arise, and the natural "interest 
in comparing, alternative hardware and software modules wit,hin each 
^system i t6elf ,* necessitate a comprehensive cost structure. To 
compute over-all average costs, while a simple and certainly a useful 
procedure, would result in too gross an aggregation for our purposes. 

Thus .the structure outlined in this section reduces the two . 
systems to tl^e "component" level, for hardwate, software, and course- 
ware. We also single out administrative and otit-of-pocket costs. 
This analysis is particularly important since changing to* perhaps ' ' 
similar "but alternat^ive approaches to various components will possibly 
occur in the current system implementations and will very probably 
occur, in follow-on systems, A further concurrent distinction is 
provided fso separate developt^ental, capital, and operating costs. 
These differentiations are ess^ential f or . predic tive purposes. 

The cost-collection systems outlined for TICCIT and PLATO have 
been made as broadly compatible' as possible. Each CAI system, however 
has a unique approach and special, circumstances 'that must be accounted 
for. For example, the' Illinois project has been in operation for 
a much longer time than the MITRE system^ but it woulji scarcely be 
.beneficial to attempt an historical account now of detailed develop- 
"mental costs for* the Illinois effort. As a consequence, the newer 
MITRE system is likely to appear compa rat ively heavy on development 
costs. To give a^'fair picture of the economic effects and potential 
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Of each of the two CAI systems, developmental costs will be^'system- 
alically separated from operating costs. Itshould be. clear* that 
a gross average of total cost divided by* some unit of visage would 
give, in this case, an unrepresentative view of the tru.e situation. 

The^ cos t--analysis team must provide accounting mechanisms that' 
■operate so smoothly as to minimiJ:e the possibility of disturb^ihg^ 
the participants. Fortunately,' the MITRE, Brigham Young, and ^llijiois'^ 
projects have administjrative personnel who are very capabJ^e in 
financial and accountljng matters. Although the cost classifications 
presented here have* not yet been formally reviewed by these adminis- 
trators, 6br review ofi the existing accounting mechanisms indicates 
that implementing, our |plan can be accomplished with the requisite 
smoothness. The area$^where precise dat^a may^e lacking are indicated 
in .the descV ipi:ion of the cost structure, and methods of es'timatiori 
are^noted. In most cases^ however, cost data by the categories 
described are. already being collec*ted by' the participants,, on their' 
own initiatlYe, or we haye discussed Jihem with the participants and 
reached the understanding that they can be collected* 

Overv iew of Cost Str u cture 

During the next year the cost -analysis team will assist Illinois 
and MITRE in accumulat ing-- d'e-vtelopment costs within four major categor- 
ies: 'hardware, software, courseware, and administration. The team 
is well aware of the many differences between the projects, not only 
in syst<jias concepts but in. implementation methods, length of project 
life, organizational structure, and the like. Despite these differ- 
ences, howevejr, it is felt that common categories can be worked out 
that will be consistent and compiarable. ' ^ ^ 

A. Hardware 

Both projects rely upon purchaged oof f -the-^helf cd'mponents for 
the^ sigjiificant part of their system. These components are 
either used as is, or* are modified by project personnel for 
Special use. Both projects^ should , therefore , be able to accum- 
ulate the purchase or lease costs of equipment, and accrue the 
costs of modification through the personnel accounting system. 
For uniquely .developed components, a unit cost will have! to Ije 
established. * * 

B. Software • \ 

Both projects are investing heavily in special ^communications 
software, the handling of . terminal interactions being one of 
the major tasks of the communicat i^ons processors. These 
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developments are beings-conducted 'by th<?' project teams' and can 
be captured as part of the data collection requirement* ' In 
other instances there is modification of vendor-supplied soft- 
ware which ,is subject to the same consideration* 

t 

C. Coursew'are • ' - . 



^The development of course materials is, in a sens-e, being sub- 
'cpntracted by both projects. The , MITRE project has centralized 
its coAiyseware effort at Brigham Young University. The project 
costs can be. ^ closely mon ito red with lictle difficulty. The 
Illinois project's courseware effort is decentralized, involving 
teachers arid pther staff of the several schools and colleges 
where the PLATO system ';ls to be used. Gathering these costs 
will require the special attention and cooperation of all per^sons 
and schools involved, if we are to get a comprehensive picture. 
A subsequent, sect ion of this report details how data are to be 
collected for the courseware. 

, ' \ * . . ' 

Da Administration 

Both. pro j ects are subentities of larger organizations, i.e. the 
MITRE Corporation anxi the University of Illinois. Although 
^ the , two institutions' operate for different purposes and with 
different types of bookkeeping, the aBminis t ra t ive structure 
of the projects includes all of the normal personnel and over- 
head costs that) one would expect to find in that context. We 
are confident that the individual's who cai^ry the administrative 
burden of the projects can meet our requests for comparable 
administjative cost data. 

The principal contacts for gatheriifg cost information are: 
TICCIT Project • . 

MI^TRE ' Ned Burr 

Brigham Young Sttiart Low c 

Northern Virginia ^t^^ 

Communit^y College To be determined , . 

Phoenix Marvin Lassila (Maricopa County 

Community College 



District , Comptroller) 



PLATO Project 



CERL/llli]r{ois Frank Propst 

Chicago Community 

College System Donald Hill (Vice-chancellor, Fiscal 

Affairs) 

Parkland Community 

College o David Johnson (Director, Learning 

Resource Center) 

Urbana School ^ 

Distr^t To be determined , 
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E. Site Development' ^ . ^ 

Eaci: project will have :to modify the sch.ool ' s i'te to accommodate, 
the terminals and communication equipment.- These a,e one-time 
.costs and will,b''e obt/ined as, they OTgcur. ' 

Cost Categories CAI P rojects . ^ , \ - , 

The cost categories descrlted below have h,e^ja~^s:lra-brllshed ^ f or 
data collection .a^ MITRE, Br igham yourig , and I14.inois. None of the./ 
categories is described exhaustively, and it is' possible thaw s5me 
of the p-^r-t'icular pieces of hardware or software have been superseded 
since our most recent visits. »Our purpos'e 'at present is to expia^in^ 
th*i structure of the cost-^collect'lon i^ystgm rather than each specific 
detail, 'but we* have tried to provivie enough ref)resentative detail to 
clarify i^ntent. - , ' ' ' 

.Each of the projects will'have cos>.s collected^ into the <ive 
general c^ategories of hardware, software, courseware,, adm^^is trai;ion , . 
and incremental site (out-of-pocket) tfosts. These five o^^legories 
appear to be natural choices. They form a comfortable' way of concept- 
ualizing the projects,* being easily understood by CAI eixpe^t^ a§ 
well as those le^s knowTe Jgeable ; they provide the cost-analyfiis 
team with the type of data sought ; - arid they are allj.eady being used / 
to^ some extent to organize efforts and\, per sonjiel , thus simplifying 
our cost collection. \ - , • - 

- - . In each of these five catego?^es and in the respective sub- 
categories, we plan to distingqish-among. develdpment (on^-time) costs,"' 
implementation costs (one time for each site), "and operating costs 
(recurring). As our experience with the project growd,. it shoujd 
become p/rogressively easier to determine where costs f^ll in this 
three-part classification scheme. ' ' . ^' 

It is evident also that the question- of allocation of fdxed- 
capital outlays will no^t-^be 'simple to resolve* In this discussion 
of structure we make no' attempt- to not'e either vrhlch co"5ts are of 
this type or which way to^ allocate taem, but simply record our need 
to contend with the question. j ^ 

, * ' ' » 

A. MITRE Project \ ' ■ 

/ .... 

1. Hardware 

Hardware development costs have been divic^ed into- five, 
, ^ subcategories. • We shall accumulate information on dollars 
expeaded, using purchase costs for off-the-shelf items 
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and development costs for MITRE-produced items. The MITRE- 
produced components will be difficult to cost evaluate for 
two reasons. sPirst, the present level of detail may not. 
be -S4if f 1 cient to permit our getting "exact figures, and 
second, the qua'n tity^ produced may be so small ^is to reflect 
ah abnormally high cost. We hope tha,t discussion with 
involved personnel can help.enoughv to' prevent thi^s, potential 
problem. We £re also aware^ however, that Vost of the 
^ MITRE equipment is composed of off-the-shelf items with 
specific purchase prices; thus, th^e cost problem o^.in- 
house producVdoh will not be great. 

Hardware has been separatee! in-to four' parts according to 
the function of specific pieces of equipme^nt; a fiftK 
category covers hardware-i?ela^^ed^ personnel costs. 

a . Proces^sor Facility 

In t,hld' subcat e'^gp.ry axe included all the equipment 
normally assotxlated with *'the operat4.on of a'.primary 
central processing unit in a computing centet the 
main ptoce^^or^ (a >I)ata General NOVA 800) ''in this Case, 
, plus any directly associated peripheral equipifient 

such as nard reader, printer, magnetic tapes and drives, 
cburkeware and data disks, and swapping disks or drums. 
It is our current" understanding th-.it all of these 
items are to be purchased as standard equipment from 
^.in'anuf acturers , and th'^t each community college will 
' require^ a full complement of this equipment. 

b*. Communications' System 

Under this subcategory is grouped the associated hard- 
• ware required to transmit messages from the central 
site to the^scudent terminal . Some items such as the 
communicatio^s^proces-46r (also NOVA 800) and the 
linking disk are^readlly available, with manufacturer's 
prices. Other^ items such as the ch a r^acter generator, 
video refresher , coloil enqoder and switches, and audio 
switch are also included under jcommunicatlons . They 
are all MITRE-des igne(j ahd, excep^^o^rthe video 
refresher, will probably all be MITRE-prbduxjed . The. 
fact that all fall under the same subca tegory^s^m^ha t* 
simplifies the co^t^collection task. ^^^^ 

The items included under communicat iono could conceiv- 
ably fall into ^th.e^r. 3ubcAt_€tgories (linking disk as 
part of the processor facility, color encoder as part 
of the terminal), but we felt that they were all 
performing the distinctly identifiable function of 
communicatic IS chat should be separated from the central 
facility and the terminal itself. » 
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c. Terminal c 

This subcategory represents- the par.t the stu^dent sees 
directly — the TV set and keyboard. An illustrative 
cos.t problem that arises in .this Connectioa is that 
selected parts and capabilities" ot the standard TV set 
may not , be necessary fpr CAI and may have to be removed,* 
thereby lo\^er/ing »^he theoretical but not currently 
actual cost of the terminal , . > 

d. ^ Audio/Video an^ Other ^ - 

The audio playback units and' the video playback units 
will be costed separately, since it s^ems conceivabT.e 
that a ' "stripped-down" sys.tem could function with' a 
subset of courseware modules without these devices* 

e. Personnel* Cost s 

P^-rsonnel .costs consist of* the professional engineering 
staff and their supporting clerical assistance, class-- 
ified by MITRE to be^working on TICCIT system. 

We expect the normal activities of modifications and 
repairs to .fall into thig category. ,^,Note that the 
personnel cost of MITRE-fproduced hardware component s 
is included Xii O'ther (notably communications) subcate- 
gories. It is our conjecture that a substantial ' 
personnel effortmay be expected to assemble, integrate, 
and test all, of the various hardware components to 
make them woyirk as a unit, since many were produced for^ 
non-CAI puriioses. Such costs will be included here, 
if they do occur. 

2\ Sof twar e 

Software development and maintenance, costs wi''l consist ^ 
primarily of personnel costs, unless additional computer 
time has to be purchased from an outside time-sharing 
vendor. Costs will include, \but not necessarily be broken 
out b^, design, coding, implementation, debugging, and 
testing of all the many necessary programs. Following the 
natural organization of MITRE, we divide the software costs 
into three subcategories: 

a.^ Application Software ^ 

This subcategory includes graphics routines, response-: 
parsing routines, programs to track performance, learner- 
and ac^ /isory-control programs and the* data formats on 
which they m^y operate. These programs in a more 
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conventional system might be thought of as the "lang- 
uage" that would provide t,he nfecessary facilities 
and functions to a programiher /author . .However, MITRE's 
approach is unirue, 'and we feel it* necessary to moVe 
the* basic application functions into this subcategory, 
but to consider the "authoring" software separately\ 

b. Authoring Software 

.This subcategory consists of the developmen^t of programs 
to "pre-process" course material from spme prespecified 
format to an ALGOL computer program. Thus we need 
costs" for any ALGOL jcompiler modifications, but more 
importantly for the development of an on-line source 
data entry system. .The uniqueness and promise of 
mitre's attempt to -free authors from computer program--, 
ming warrant monitoring that development separately, 

c. Delivery Software 

Despite the f^ct that Data General supplies a standard 
operating system (RTOS) , it probably will have to be 
modified^- to cut out some of its general, unneeded 
'functions, and to add others peculiar to this CAI project 
We would also include here any programs or subpxog'rams 
necessary to process student records and to operate 
the terminal processor. 

3. Courseware 

Courseware costs are being accumulated at Brigham Young 
University according to a format worked out by the cost- 
analysis team and members of the project staff. The following 
table lists the various courseware development specialists. 
Costs/would be accumulated in dollars-per-hour according to 
the time spent by each of these specialists to prepare a 
base frame of ins tructional materials per course during: 
1) the initial .stages of the project, 2) the transient 
stage of, tlie project, » and 3) the final, or steady-state of 
the project. The purpose of this type of analysis is to 
display the "learning curve'' aspect of courseware production 
that is characteristic; to the activity.' The information 
would be among the most pertinent for future ushers of this 
type of lauthorinfe system. It is anticipated that time and 
costs wi'll be gathered monthly pn a course-by-course basis. 
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TABLE I 



Differential Hour Accumulation Table 
BYU Coursware Project * 



Mathematics I 


$/hr 


Initial 
Hourfe-Frame 


Transient^ 


Steady- 
State 


Instructional Psychologist. 










Instructional Design Technician 






■> 




Empirical Desi^gn Technician 










Packaging Specialist 










Author 










Program Implementation 











With the exception of Program Implementation., all course development 
costs will be collected at Brigham Young Univers/ity, Program Imple- 
mentation costs will be gathered^ at MITRE. 



4. ■ Administration 

\ 

The costs, related to' resources and to individuals concerned:.. 
with the ' administration of the ,cours/ewai|e development at | 
BYU will be accumulated as 'a sepa^a^e category. These costs 
will th en be allocated l.o each course monthly on the basis': 
of the ratio of each course-developiient cost of the total j v 
developmental costs for the month, ^e.g.: 



A, - - 
i m 



- A* 



LCH 



A^ = Individual Administration Course Costs 
CH^ = Developmental Hour Costs in Table I 
I « the sum of all D^^velopmental Hour Costs 
A* = the monthly total -jf Administration' Cost s 
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It is assumed these costs will primarily include. Mr. Low's 
salary, secretarial posts, and those of occasional admin- 
istrative assistance support . Working with Mr . Low, we 
intend to isolat.e requisite clerical costs for the operation 
\ofeachcourse. , 

5, Incremental Site (Out-of-Pocket ) ^ Costs ^ 

These costs include expenditures wh±ch mus't be made to 
•adapt a site to the technical requiremrenf s 'of a TICCIT 
system. They wouTd include such "'facilities preparation as 
air conditi^^ng, electr^gal .wiring and fixtures, and 
facility modification. /Other operating costs likely to 
occur are telephone-Jih^ installation and rental. Any 
other significant costs >hat can be IsolatedWiH also 
be included, posslblV sulc^ as space cost. 

Illinois Project ' 




Hardware 



Hardware costs for the PLATO project have been divided into 
the same five subcategories as fo?' MITRE. We shall accum- 
ulate dollars expended , using purchase or lease costs for 
off-the-shelf items and development' coVt-s^— £pr items produced 
at the University of Illinois. The specif i^ items 'under 
each heading, however, could not be mor.e-'dif f erent for 
TICCIT and PLATO, reflecting as Trh^y-<?d^ the very different 
approaches of the two CAI proj ects /^-Such differences will 
thu9 make direct comparisons of subcategories mi&leading 
and, by and large, inappropriate. 



a. Processor Facility * 

This subcategory is primarily the rental of a standard 
CDC 6400 with • it^. large extended core storage. All 
disk units arev included under this heading. We are 
not aware of any ^peripherals , such as card reader or 
magnetic-tape drives on the PLATO system*. \ We have also 
put the peripheral processor units in this\ subcategory 
since they are an integral part of the CDC >400 (even 
though one could view th^eir f unption as communication). 

b . Communications Sy^stdm 

In this subcategory we have the main network Interface 
unit, and ;site controllers. This latter equipment iy 
produced and maintained in-house . There is likely to 
be included 'here.,^ relatively large telephone-line 
(coaxial cab le) 'charge since the terminals will be 



\ 



c 
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driven from a single central site at the linlversity 
of Illinois rather than from a local processor. 

c. Terminal ^ 

The plaCsma display terminal used by PLATO was invented 
and developed at/ the University of Illinois and is 
, now being prc'tu^^d^ommercially ♦ It entailed a 
sul;^ tatitial deveuo'pG'ent,al expense over many years; 
we shall ^ot collect these^costs. 




Tlie terminal al^o comprise ^ a ^ keyboard, a touch panel, 
.and a ml^c^ofiche slide projector that operates under 
computer N^ontrol ♦ . 

d* Audio and Other r 



i-acle 



The random-access computer-controlled audio unit is 
in this subcategary* At this time it is .the onlyj* " 
separable unit, and the system\^an be operated ^^i^hout 

e. Personnel Costs 




These costs will primarily be thflse associated with 
installation and maintenance related to hardware ^ 
particularly for the main computer, communica t\ons , 
and the plasma terminal. Since PLATO has been in 
existence for several years much of the integration 
work has already been done, but the process of developing 
maintenance routines and documentation will incur some 
personnel expenses. 

2. Software 

Following CERL's natural organization, software is divided 
into three subcategories; these are quite different from 
mitre's, owing to the najture of ;the approach. The software 
cost is primarily personnel costs of design, codi^ig, imple- 
mentation, debugging, and testing. Since a large number 
of programs are already written, a large segment of the 
costs •collected will be for additions and improvements, or 
^ conversion from PLATO. Ill to PLATO IV. 

f.. Authoring Software (TUTOR Language) 

Development and modification of the main authoring 
language now widely used on PLATO are under this 
heading. The editing routines would also be included 
here, as wel4.\ as the proctoring program, since these 
serve an authoring function as opposed to a systems 
f unc t ion ♦ ' • 
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b. System Software (Operating System and Motiitor) • 

This subcategory include any modifications necessary 
to the standard ope rat In g system for the CDC 6400, 
such as stripping awayj t4ie unn-eeded'^ magnet4.c ' tape 
functions. It may be thkt further c^hanges will Ije 
required as la;:ge numbers of new ^terminals are 'added 
to .the system, \ 

c. Courseware Support' ' 

Under this subcategory we have included any routines 
such as the master progrr.m that manage tire' Less'on 
material and keep track^of student records and, the like 

C 

Courseware . * ^ 



to4^rseware is to be developed "^with Illinois staff assistalice 
In the- Chicago Commui>ity College System and at' Parkland » 
Community Colle'^fes*^ Specific faculty members will be siven 
released^ time for' one year or longer on a half- or f ull- 
trm^ basip to develop course^material^ in TUTOR oii the 
^^IXto system. This'method contrasts' markedly with >the 
TICCIT effort cei^tralized irndfir Professor Eunderson at- 
Brl|ham Young uYiversity. The lack of ceAtra^ized effort 
maty^^ake cour'se /costs s'omewhah more difficult^\o follow, 
but there are P«\TO ^personnel assigned . to coordinate this 
effort who can Issist with the accumulation of costs on 
a course-by-course basis. ' 

\ ^ ' ■ 

Courseware costs will\be collected in the following two 
subcategories: \ 

*a. Direct pJculty Costs 

\ • 

We intend to divide the processof course development 
into four distinct stages: y 

1) Time to learru TUTOR' 



2 ) Time to develop a module , including planning 
time and time on the PLATO terminal 

3) Time to, test a course module 

4) Time to document a course 



'While such analytic data may not be readily available, 
we find 'the effort to produce the, data ^es^sen^ial* ' 
Previous pAI systems have come under criticidi;. for not 
being oriented to .f f'cnilty , * and , it seems that the PLATO 
sys-^m has. the pot en «>44l» f or avoiding, such problems. 
This fact needs to be cost do&ument ed ♦ ' 

' ; ' * ^, 

. We expect vo develop time-recording *probedures for. tHe 

^individual instructors In the cd^imunity coilegeJ'.', In 
addition, we-ha've established that if is feasible to 
track faculty use of the system with "PROCTOR." ,At 
present we liope to accumulate automatically all console 
time on a course, and, using the tfime records, .to key v 
JLt to off-fystem effort. \^ 

b, Author-Support Costs, ' , . ' 

We group into t\is subcategory all non-personnel costs 
associated with writdi^g a cou*rse -- such as photofer.aphi 
recording, drafting, art ' work, publications , and ^ 
library-support costs . These costs will be recorded 
^at the CERL as a separate charge tp. each course. / 

4 Administration * / 

-/ 

We intend to follow the same approach uared with the MITRE 
syetem. Administrative costs wUll be more- diffuse; the 
community colleges as we 11. as Illinois are expending 
administrative effort. The development of an adequate, 
compre^hensive system for 'administration will be on a trial- 
and-e^or cojiirse in the coming months, but the costs to 
expectyseem jpbvJ.ous . Initial general categories are faculty 
administration , student' administration, and clerical . 

/ 

5. Incremental Site (Out-of-Pocket:) Costs 

-These costs include expenditures that must be made to 
adapt a site to the technical requirnme;its of the PLATO 
system. Since the central processor is to remain at CERL, 
extensive site preparations will be avoided. Installation 
of the site controllersfwill be included here , as well as • 
installation of terminals. Other operating costs likely 
to occur are local telephone-line installation and rental. 
Any other costs that can be isolated will also be included, 
such as possible space costs. \ • • 
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Cost -Categories - ^Baseline Course Costs for Community Colleges 
• 

The baseline costs o£ education in a community college environment, 
as they are .perceived* by car reixt ''financial data systems us'ed in most; 
schools, consist -of faculty compensation,^ si^^ce , departmental costs, ^ 
student expenses, and duxili^ry costs. For some coursea, such as math, 
these auxiliary costs may not be important, but for English and the* 
^ience courses they are supportive resources which must .be evaluated 
economically with* and. without the CAI investment. A 8*econd important 
consideration /is how the teacher spends time with and Nwithout CAI. 

^We- intend, in 1972-73, to gather data which indicate on a percentage 
basis how an instructor spends'time in the schools selected as proto- 
types for the CAI project. This^^is a delica^t^^ process, as we need < 
th'e instructor'^ cooperation for all manner of " data collection. Our 
intent is to develop anonymo'us measures of s^evetal individuals, 
within the project and/without. Compensation may be provided to 
instructors for their assistance in filling-rout time-analysis forms. 
The initial f6rms will be simple, developed with the teacliers in order 
that they understand our* ob j ec tvies . The base costs for the . community 
college insf^ucto^r activity probably have s-ome situation-specific 
elements, but they alao include general factors of teachi^r behavior 

•^ts^h^jt can be verified at other institutions. ^The plan is' to develop 
the* community college base' costs in the PLATO project and if neces- 
sary refine th^m i'n the MlTRE demonstration sites. -^t 

In all of the following cqst areas, the basic item of information 
sought is the cost of teaching the individual student under preseiit 
methods in a given course. * ^ 

A. Direct/Faculty Co8j:s.. 

" / ' ' ' 

In every educational situation today, faculty salaries are 
by far^^the largest item of expense. In most instances faculty 
members' are paid either on an annual (school year) basi*s with 
a minimum teaching load being described, or on a semester course 
basis, in the case of part-time teachers. In each instance, 
the instructor's efforts are described in terms of the amount 
of time he spends in class. In some^ cases faculty are paid 
- ' additional amounts for c'ourse preparation or for attendance 

at workshops and* conferences that relate directly to the cO^urses 
being taught. The dire*ct faculty costs that will be gathered 
for baseline analysis will be: 

^ 1) Teaching hours the. average and total hourly- cost 

of teaching tlje* courses being prepared for CAI. A taxonomy^ 
that sorts tixaei into classroom, counseling^ and grading, 
will be the starting base. It is assumed that experience with 
the teachers will, allow us to refine this taxonomy. 
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2) Hours of preparation 

, * / '» . 

3) - Total. hourly cost o f * out 70 f- r. lass ^ paid time for 
courses being prepared for CAI as noted under Co.urseware 
Development for normal class preparation, including library 
work,, correspondence, and graphic or hand-xy.ut preparation.. 

* * ' 

B. Allocated Space Costs 

The spac^e costs of education in the community college will be 
collected, to include: * ^ \ 

- ' : tr 

1) Average classroom costs' foi^ coui;ses being prepared 

. for CAI. ^ . ^ 

2) Average cost of any other' special study space used 
for these courses, * 

Direct Departmental Costs 

The^verhead costs of running community college departme'nts will 
.be added on a i>er-course -basis. The auxij^iary library/^nd 
laboratory costs will be included in this category. Primarily ' 
the co^st will be administrative time to plan -and- implement the 
coordinat ion.^of ^udents to fax:ilities, in order to provide a 
course of instruction. ,In addition, vegistratioh and record- 
keeping costs will be accumulated. 

D. Student Costs 

Although rarely shown as a cost to the community college, there 
are costs to the student under the preg:fent method of teaching ' 
that could be directly aff'ected by^ the use of CAl/ An aspect 
of this tategory is t*he informal activities . of the student in 
campus unions or off campus. We intei;^ to work with the school 
administration ^nd a committee of students to deVelop methods 
of accumulating data on these activites. Because we have not 
yet had an opportunity to discuss our needs with individuals 
at, the demonstration sites, we are unable to be very spec-ific 
about the form fn which these data wiJI be recorded. Thus at 
this time we propose ortly to collect two items of student costs 
as part of the baseline cost study, although we assume the 
cost system will be modified with* experience: 

1). Textbooks and standard fees (if ' any) . Most courses 
require the student to p^y for materials that are to be 
retained, and occasionally for the use of special equip- 
ment or services (such as computer time) in the school. 
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2) Class time (or time to complete, a course). Many , i 
students, In community colleges are working at the same / 
time, or ar«' in school only until they xan complete a 
particular program so that they can enter a profession - 
o,r tfsade. For. that reason, there 'is an 1 oppor t un i t y cost' 
to the student if the .course of 'study take^s^- twice as long 
by one method than by another. 

The cos t-anAlj^ois team anticipates that these cost categories , 
will be refined and ^expanded , as necessary, when closer contacts 
have been made with all of the community colleges^ participating in • 
the CAT p-rojecf. Our feeling at present is. that baseline icost data 
of an uncomplicated sort should be ^ the' goal , * Viost educatcirs who 
are cons-idering the CAT method of teaching either as an addition to 
their aurreht program or a^e a r .eplac tjment ^f o r classroom teaching 
are concerned about the comparison of one method with the" other ! 
on a cost-per-s»t!udent-hour basis, and that is .what the cost analysis 
is designed to provide, * • 



T 
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EDUCA'^ONAL ANALYSIS • ^ - ' 

* • * 

This section describes the baseline educational analysis that 
will be implemented during 1972-73. ' Attention is given to achf-eve- 
ment and attitude measurement, the experimental design, and plans 
for data collection. 

.The discussion of achievement measa^t,ement includes a rationale 
for using standardized tests as a basis, ^'or comparison and for using 
item-sampling ,and multij)le-mat rix sampling techniques to, obtain group 
measures. A discussion follows of the J:echniques for attitude measure- 
ment, with brief descriptions of survey^^'and interview procedures 
projected" ^or the coming- .year . A final section details the experi- 
mental design and .our plans for baseline data collection', ' it also 
summarizes several impi^^r tan t questions that must be considered 
before we can estimate the' effect iveness of computer-basfed education'. 

Achievement Tests 

• Standardized achievement tests offer a number of potential 
advantages for the evaluat'ion q f ^ins t r uct ional programs. Shoemaker 
(1-972) lists four adv^^^ntages : • no developmental costs, relatively 
low. cost, develdpment/by professionals, and availability ' of validity 
data. Broadly in terpj/re ted , the last of, these advantages is of 
particular importanc^ to an evaluation. Prior information about the s 
psychometric properties of the instrument can insur.e . faat the measure 
has sufficient reliyability for the intended use, that it is app ro- 
priate in terms of /difficulty level for. the group, in question, and " 
that it has- validi/ty for the intended purpose. . ^ 

Normative data aA^ailable for standardized tests provide an 
additional basis of' -comparis^in for new programs. Al'though such com- 
parisons are subject to many pivfaLls, they do help in^ the communica- 
tion* to a wide audience. 'Performing very' well on a well known standard 
ized test provides additional credibility to claims of program success 

more ^than ca'n be achieved by. comparison^ within the evaluation 
study, usin,g unknown instruments. 

Th£ normative comparison value of standardized tests will be 
particularly important in the evaluation' of the elementary school 
component of the Illinois demoiist rat ion . Whatever additional 
values^ a ^^ogram may have, there is a need to provide public assurance 
that elementary school students av^i achieving adequately in the lireas 
of reading and arithmetic as reflected by scores on the standardized 
achievement tests. 



Alth ough ^Standardized tests will play an important 1 e in the 
evaluation of the MITRE and Illin^ois demonstrations, it would 
be unwise to depend only on them, . ^here are disadvantages as weli 
as advantages to using standardized tests; thus we plan to complement 
the standardized tests with bther testing procedures that are not 
subject to the same disadvantages. In partic.ular we sliall a-tteinp^ 
to use other testing procedures to overcome two important limitations* 
of sole dependence on standardized tests. These limitations are 
related to test length and to content coverage, 

\ 

Since mos,t s tandax'dized tests are designed for individual assess- 
ment they are longer than they need to be for purposes of group 
comparison. The basic concern of program evaluation', however, is ^ 
with group comparison rather than individual assessment. The unne'c- 
essflfry test length results "either in an unduly ' heavy burden on ^ 
students and^, teachers .in terms of testing time or it limits the 
evaluation to ..a narrow domain of test content. The former alternative 
can nesult in resistance from stiideUts and teachers, and the latter 
can result in a failure to detect important program strengths or 
weaknesses- i>n areas no^t measured. For a given amount' of testing time 
there is a trade-?off between fidelity (i,e., the accuracy, of measure- 
ment of ,a given dimension) and bandwidth (i,e,, the number of diff- 
erent dimension^ or component skills that ^are assessed) (Cronbach & 
Gleser, 1965), \Since jLess fidelity is needed for group comparisons 
than f or indiviaual measurement" we plan to 'supplement tfte core sTrand- 
ardized tests with ^short tests using the item-sampling tdchnl^ues 
discussed^ later' in this section, • • ' 

As noted, a major disadvantage of standardized tests is related 
to their content coverage. In order to be widely applicable, standard 
ized tests mast be fairly^ general ifi nature to* insOre their appropri- 
ateness to many different" types of instpuc tional programs, Tet as 
Hartnett (1971) has noted, making a standardized test broadly applic- 
able may also make it insensitive to important prpgram differences* 
in .more ' specif ic optcomes,^ According to Shoemaker* "the primary' limit- 
ation [of a standardized test] is that such a test is not likely to 
contain both the. breadth and depth of content coverage necessdry to 
make a detailed assessment of any instructional program. This insens- 
itivity of the test is likely to lead to the conclusion ot 'no diff- 
erence^ among programs having distinctly different characteristics" 
(Shoemaker, 1972;, p5) . . ' ' 

As we have indicated, we plan to complement the standardized 
tfests^by item sampling .:ind possibly ^multiple-matrijx sampling tech- 
niques in obtaining group measures. In item sampling a short test 
that is n items in length is constructed by random or stratified 
random sampling from a pool of n items inhere n is greater 
than n , The short n-item test -can then be used to obtain estimates 
of mean 'performance for the pool of n^ items (see Lord & Novick, 1968 
PP 253-254), For^purposes of evaluation, however," the estimate of 
the mean on the n . ppol is^less important than the simple comparison 
between the means of the demonstration and comparison4 groups on' the ^' 
n-item test. ^J 
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An aieern'ative to halving ea.ch student take ^eyeral sli^ort: test^ 
would be to have samples of students respoHSd to different longer 
tests* For example , t.ests in five areas were needed, each consisting* 
of 100 items, a choice might be made between giving a samp-le of 20 
itiems from each test 'to aj-l examinees or giving one of the 100-i-J:em 
tests to non-ovejL-lappi'i^^sampPes of the students. The relative^ 
.strengths of tiiese two approaches may be contrasted by considering* 
the sampling variance of mean proportion, o f i'^tems ^nswerefi correctly, 
z , which can be written ^ - ^ - 



^ nN , . , 
g=l a=l 



8 a 



where yg^ is the score for examinee a on item g^ ^yga ^ if correct, 
and 0 if wrong), ^n is -the num^ber of items and N is the number 
of examinees. If the. n items are' a sample from a larger pool^of- n 
items and the N examinees are a sample from a population of N 
examinees, then the variance of z given in* Lord & Novick (1968, 
equatio.n 11.11.6) is a function of n , n , N ,#N_^, a| , a| , and Z. 
The capital Z is used to refer to scores of all n* items in "the pool. 
Thus Z is the mean proportion right for the N oxaminee's on the 
n^ item pool and is the variance of the n Item test. The t^rro 

Op is the variance of the difficulties qf the n items. Formulas 
for., estimating these terms may be found 'in Lord & Novick (B968).. 

The critical point for purposes of the p'resept discussion is not 
the actual formulas but the implications of manipulating; ' n and2-N 
on the size of the sampling variartce of the estimate,' z . If 
is large relative to ^ as wirll g.eneraliy be tlie case; the variance 

of z will be decreased if n is reduced and N-^ increased, holding 
nN constant. Tlu^, for a constant nN , a better estimate of the 
group mean performance would be obtai^ned by sampling it^ms than by 
sampling examinsees. 

/. 

A more complicared but also a mor^ powerful approach is that of 
mult iple-matrix sampling,' In multiple- matrix sampling, diTferent' • 
sam[p>les ^f items from a common item podl are administered t o ^dl f f erren t 
samples of examinees. An e'f^.'lcient multiple-matrix sampling procedure 
would be to 'divide the item podl ^q^f n items into ^ no.n-o verlapp ing 
random samples of n to be administered to M non-o verlap^'ing 
random samples of N = M/M examinees. This approach c^n bi shown to 
be superior to either item or examinee sampling-, '(see Lord and Novick, 
1968, pp 255-258) . 

* . . . • 

The use of^ multiple matrix sampling for the evaluation of^^instruc- 
tional programs" has been strongly advocai^d by Shoemaker (1972r. As 
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Shoemaker acknowledges, however, the multiple ^matrix approa'cb can 
present serious problems of logistics since the different administra- 
tions of different subtests to different sub-groups will frequently 
require different oral instructions thus requiring greater disruption 
of normal classroom instruct! n. This difficulty has led us to elect 
a strategy causing standardised tests and simple item sampling for 
most of the testing that would be common to ^he comparison groups 
(baseline year and control groups during demonstration) and the 
demonstration gro4ips. 

The' multiple matrix sampling approach can be used very readily 
with the CAI students. Therefore, we plan to make use of this approach 
in some of the on-line testing that will be accomplished during* the 
evaluation. • 



V 



Attitudes and Activi ties^ ' 

Not otily will these demons txat ions produce changes in student 
achievement and cognitive skills, they will also quite likely affect 
the attitudes and ac ti vit ies of students and teachers. Indeed, the 
principal contractors have argued that attitudinal changes may consti- 
tute the most important ' effects of computer-based education. Attitu- 
dinal changes will, of course, be indirectly reflected in student 
acnievement; we intend, however, to gather direct information to 
help us assess the influence of this mode of instruction. 

It is important in. projects as large and complex as thes^ to 
determine how students react to the several components of their exper- 
iences. We should therefore measure not only attitudes toward specific 
courses*, their subject matter,' methods in which they are presented, 
grading systems, and appropriateness of initial placement of the student, 
but also alterations in faculty-student and student-student relation- 
ships, and attitudes toward studying and toward computers and computer- 
assisted instruction. A course that achie\^^'^j its educational objectives 
but liaves the student feeling more than ever deperson.:ilized or 
desirous of leaving school has gravely failed in its purpose. On 
the other hand, we may find a student's general attitude toward 
computers or studying significantly improved even thdugh he ^s found 
the course material inappropriate. Such outcomes will require the . 
most carefu^ anaj.ysis- 
f 

Student activities v?ill also yield information about po^ntially 
important educational effects of CAI. The number of times the student 
chooses tx> sign, on to t.he system (this beipg related to hi;, class 
attendance) , ^the ncrmber of extra options he attempts, and the latency 
of his responses are relevant computer-related activities. We shall 
want to know how much time the student spends in study outside the 
classroom, how orten^> he uses the library and other sources of supple- 
mental information, and how frequently he has voluntarily contacted 
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faculty members or other, models for stimulation and assistance. 
Similarly, we shall wish to assess the student's general educational 
progress seeking information on whe^ther he drops the course or per- 
sists in -it, and whether ^he does or does not take follow-on courses. 
Such data must be considered as noteworthy behavioral effects of 
this mode of instruction. 

During the baseline year, the primary metho-d, we shall use for 
gathering data on student attitudes and activities will be to admin- 
ister, to samples of students at the end of each term, parts of the 
following standariized attitude surveys: 

1. Student Instructional Report (SIR). 

This survey was , developed to give faculty members an oppor- 
tunity to have their students describe and assess their courses 
and instruction systema*t ica'^.ly and to give students a chance 
to express their views of the course and the way it was taught. 
SIR will, of course, furnish the same information for the 
evaluation team. Since 1971-72 was the first operational year 
for this instrument, comparative^ da^L^are not yet compLetely 
.available, but they will be r^ady wheiy^ needed . There are 39 
items dealing with subject matter, td^ching methods, materials, 
and I • udent-f acuity relationships; there is also provision for 
ten additional items dealing with issues specific to thi's ^ 
project. 

2. Comparative Guidance and Placement Program (CGP) Student' 
Questionnaire: 

This survey asks students about their satisfaction with' their 
courses, their major fields of st.udy, and faculty-student* 
relations. It is part icularly^ appropriate for the MITRE project 
since it asks specifically about placement in English and Math 
courses. Comparative data on 6210 students from 26 junior 
colleges are now available, and more data wijl be collected each 
year. There are 47 items and provision for six- additional, 
specially designed items. 

3. College Student Questionnaire - Part' 2 (CSQ-2) . • 

Since the instrument was designed for use at four-year colleges, 
many items are inappro,priat e for this project. However, items 
with bearing on student attitudes toward faculty and student- 
faculty relationships, attitudes toward the administration and 
other students, sources of satisfaction and p.roblems, study 
techniques and attitudes, leisure-time arc tiyities , and instruc- 
tional preferences will be combined with itejns from the following 
three sources to provide an instrument appropriate for this 
project and yet drawins upon previous work in thts field.' 
Comparative CSQ data are available for 1500 students at 37 four- . 
/ year colleges and will be updated within a few months, but these 
( data wi/JTl admittedly be of limited use since they come from 
four-year institutions. I 
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A. College and University Environment Scales (CUES). 

Several of the scales in this instrument (practicality, community, 
aw,areness, scholarship, campus morale, quality of teaching and 
faculty-student relationships) measure important potential side 
effecfs of CAI. ..Many items will be inappropriate since this 
instrument, too, w^s designed for the four-year colleges. 'Never- 
theless, a recent study using CUES has gathered some comparative 
data from junior college students and found that some of the 
items wojjered very well in this context. These will be considered 
for use/in our study.* 



:5. Student Reactions to College (SRC). 

This instrument, stillunder development, is specifically designed 
for community colleges. A preliminary form has befen administered 
at 40 community colleges this spr ing the . final form is expected 
this fall. We shall select items from SRC that are especially 
appropriate to our investigation. 

6. Non-standardized sources. 

A number of ETS researchers are developing or have developed 
student at titude^ questionnaires . We expect to use directly or 
to adapt pertinent items ,for this projecj:,. The j|;taff also will 
design a small number of items that are not available elsewhere 
and that deal with issues related specifically to CAI.. Of course, 
all items in this category will be pretested before use at the 
participating colleges. 

To prevent over-testing^ students , we shall not ask anyone ,^to 
complete all of the above questionnaires. We anticipate, inste^ad, / 
that three or four samples of 50-100 students each from courses dealing / 
with appropriate subjects will be used at each participating college l 
each semester. Thus, no student will be asked to spend more than 
half an hour completing questionnaires. / 

The second significant group whose attitudes and activities should 
be assessed is faculty and counselors at the participating colleges. 
The instructor's role will differ in the two demonstrations and, 
therefore, will necessitate different data collect'ica procedures 
in subsequent years; however, th.e necessary baseline data will be the 
same for both projects. Interest will- cenrter on changes in the 
activities of instructors and their attitudes toward certain issues 
related to the implementation of CAI. The' assessment of instructor 
attitudes should include general reactions to computers and CAI, 
scudent-facul^ty relationships and the general college atmosphere. 
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information on resoux:ces used for course materials and test questions, 
and opinions on responsibility for course content, responsibility for 
grades, individualization of instruction, importance of traditionally 
defined subject matter, uses of teyt«, and importance of defining 
course objectives. 

There will he several sources of data on instructor attitudes 
and activities. First, we have prepared a draft attitude questionnaire 
(See Appendix A) dealing with the topics mentioned above;\ this will 
be given to all faculty and counselors at participating institutions 
as soon as it has been pretested, revised, and approved. Second, small 
group discussions with 10-12 concerned teachers and counselors will 
be held at each participating college near the end of each semester.' 
Informal and open-ended, the discussion will center on the question- 
naire, the issues raised in it, and general developments in the project 
Xf these discussions bring to light issifes needing additional clari- 
fication or documentation, subsequent questionnaires may be developed 
and administered. Third, a small number of selected facul ty . members 
will^ be aske^d to keep a daily log of activities for a one week perl^od. 
Such an instrument has been develope4 by vProf essor Bruce Biddl^ at 
the University of Missouri and has hc^n used in a number of studies 
on teacher roles. Thus,, a large amount of comparative data, as 
well as a thoroughly ^developed storing sy-stem, is available. Other 
such- instruments have also been developed. We have not yet made the 
final decision about which is the test one. TKe number' of instructors 
asked to keep these logs will depend on the nurnber available at 
participating colleges and the cooperation that can be obta/ined. 
Half of these instructors will log a week in the early part of the / 
term, and half will be asked to log the final week of the term.. This/ 
process, will not be repeated each term. Finally, faculty members : 
who are preparing materials for the Illinois project will be inter- 
viewed individually at periodic intervals throughout the year. They 
wi(il be asked about their activities, the status of their relation- 
ship wi"th the developers, their expectations, problems, aiid progress. 

We fully recognize how sensitive the issues are that we intend 
to explore with faculty members. Due caution will therefore be used/ 
in soliciting cooperation from each college administration, faculty 
senate, teachers' union, and any other relevant organization. All 
responses will, of course, be held in strictest confidence, and no 
references to individuals will be made; in any reports, nor information 
be released to anyone outside the directly involved ETS staff. 

A final group of persons asked to furnish -information about ' 
attitudes and activities will be membo:rs of each collegers administra- 
tion, probably the registrar and dean of instruction. They will be 
interviewed each semester for their opinions on the relevant issues 
mentioned above. They will also be asked to furnish appropriate 
statistical data on dropout rates, popularity of follow-on c0ur9.es, 
and other such topics. ^ 
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Experimental Design and ^Data Coilec t ion Plans 
1. MITRE - Community Colleges 

Since the startl'thg cjate for the MITRE demonstrat toi{ la expected 
to be postponed Until September 1974 v^e plan to delay thW starting 
^-date ^or the baseline data collection* Ratfier^than starting base- 
line ^ata collection in the fall 1972 terra (semester or quarter) 
we plan to start with the 'spring 1973 term. This will make it 
• possible to collect 3 semesters (or 4 quarters) of baseline « da ta 
before the start of the demonstration. 

Baseline data collection could be delayed until the fall of 
M73 and still cover a full academic year. However, current MITRE 
/plans still have a target hardware- installation date during the spring * 
(of the 1973-74 academic year, and this installation might be expected 
- — to influence the baseline data collected in the spring of 1974 . a Thus, 
we thought it would be better to start during the preceding spring. 

Achievement tests and attitude questionnaires -will be administered 
to a sample o.f the target populat-ion of students at the end of each 
term. The target population will consist of all students enrolled 
in English or math courses for which equiva^lent credit could be. obtained 
from the CAI Courses during the demonstration period. For e-ach of 
the four courses (two in math and two in English) achievement test 
items and* questionnaires will be administered to a sample of. approx- 
imately 100 students each tirm. Pretest data at the beginning' of the 
course will not be collected; however, previous test scores available 
through college records^ (if any) will- be obtained. The test data 
obtained from college files, sbch as CdP or ACT scores, will be used 
for purposes of matching and'^'s covariates in the data analyses. 

♦ The "purvey of Instructor Activities and Attitudes" » (See 

Appendix A) will be administered to all faculty -In the spring of 
1973 and will be followed up by small group discussions with selected 
faculty members before the end of the school year. Further group 
discussions will be held twice at each college during 1973-74 and 
additional questionnaires will be used if needed. Administrators will 
be interviewed individually at the times of the group discussions. 

The analysis and reporting of baseline data obtained during the* 
1972-73 and 1973-74 academic years will be limited to simple descrip- 
tive statistics including means, standard deviations, and reliability 
estimates fpr each of the achievement and attitude scales. .After 
the collection of comparable data from students participating in the 
demonstration, however, these data will be used-for some of the primary 
types of comparisons. Specific dkta analyeris considerations are 
♦ discussed in a later section. 
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2, University of Illinois - Community Colleges 

The basic approach to data collection and analysis, for the com- 
munity colleges participating in the University of Illinois demons.tra- 
tion will be the same as the one described abo.ve for the MITRE ^ 
demonstration. However, the timing wljl be different fot the two 
projects, owing to their . dif ferent schedules, The^baseline data 
collection for the community colleges in the University of Illinpis 
demonstration .will begl,n with the fall 1972 term. Achievement and 
attitude data will be collected at the end of each term during the 
1972--73 academic year. The analyses of these data will follow the 
same attern as that described above for the MITRE project. 

Data will be obtained for samples of approximately 50 students 
per term in each of the five target courses (Biology, Chemistry, 
Accountancy, Mathematics, and General Education J)iploma) at each 
participating college. The students will be selected from courses 
that are expected to be partly taught by PLATO during the demonstra- 
tion years. 

The faculty questionnaire will be administered as soon as pre- 
testing and revisions are rfbmpletcsd,^ it is hoped by mid-November. 
Subsequent group discussions are planned fo^ mid-December and, April 
-- with a second questionnaire, if needed, in mid-May. CI authors 
and responsible administrators will be interviewed individually in 
the fall of 1972 and^ the spring of 1973 

Since the University* of Illinois plans to have terminals installed 
at participating community • colleges during the 1972-73 academic year, 
the baseline results could be confounded in tTi"(§' sense that some stud- 
ents will have some experience' with PL-ATO during the baseline year. ^ 
Questionnaire data will also be obtained at each institution, however, 
to ascertain" whether students have had any experience with the PLATO 
system. Students with -PLATO experience will be eliminated from the 
baseline .groups , If there is a sizeable number of students who have 
had experience with PLATO, comparisons will be made between these 
students and'baseiine students without exposure to PLATO to see 
whether the group without exposure is sufficiently representative to 
be useful as a comparison group for the main demonstration. 



3, University of Illinois - Elementary Schools 

The baseline data collection for the elementary school demonstra- 
tion will fo;,low a different pattern from the community college demon- 
strations, k pretest-posttest design will be used, with the elementary 
schools.. Achievement tests in reading and arithmetic and an attitude- 
toVard-reading questionnaire will be administered in fall, 1972. 
(Samples of reading quest io.nnaires , being considered for use are 
included in Appendix B.) The same attitude questionnaire and parallel 
forms of rhe achievement tests will be administered in May 1973. 
These data will be obtained for all students in grades 1, 2, and 3 
in the four demonstration schools. As vith the community colleges. 
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students in the elementary schools are expected to have some exposure 
to PLATO during the baseline year* Students with and without PLATO 
experience will be treated separately in the analyses. 

General Considerations in Estimating the Effectivenss of CAI -- 
Control Groups 

Th^ analyses of data obtained from quasi-experimental evaluations 
and the interpretat ions of these 'analyses have recently been the 
subject of considerable controversy, A paper by Campbell and 
Erlebacher (1970a)^nd the exchange of papers which it engendered 
(Campbell & Erlebacher, 1970b; Cicirelli, 1970; Evans & Schiller, 
1970) exemplify some of the. basic issues in this controversy. From 
the technical standpoint, t?he crux of the difficulty in designing 
evaluations of educational programs Is that the evaluator lacks the 
power of randomis.ar.ion . Without random assignment it cannot be 
assumed that groups receiving different treatments were comparable 
prior to the introduction of the treatments. The evaluator is thus 
faced with the problems of demonstrating the comparability of the 
groups prior to treatment or>^ a^s is more often the case, attempting 
to take preexisting group differences into account' in the analyses. 
A general discussion of these issues is given in Rubin (1972), 

Hence it is important to try to find no'n-CAI groups of students 
who are similar to the CAI groups of students. There appear to be 
^three collections of aon-CAI students who'^i^e potentially similar 
to the CAI students: 

(1) students from previous year, before introduction of 
CAI — "baselinfe students" 

(2) students from similar non-CAI class.es tn the same 
year 

(30 students from other institu^ons in non-CAI classes* 

The baseline students will provide one important control, group. 
The usefulness of these data will depend upon general similarity of 
student characteristics from one year to the next- and a general ^ 
commonality in purpose for courses taught during the baseline and* 
demonstration yea^s. Although reasonably good comparability is anti- 
cip'fted, it seems, desirable to augment these comparisons with a diff- 
erent type of comparison group. ♦ > 

Wherever possible, comparison groups will be formed during the 
demonstration year. Several approaches will undoubtedly have to be 
used and the details of the procedures will have to be worked out after 
the partici'pating schools have been surveyed and details of partici- 
pation specified. In cases where there are sufficient students taking 
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a course so that only part of theth can be accommodated on the CAI 
system^ comparison groups will be^ formed from students taking compar- 
able non-CAI courses. If administratively feasible, the CAI and 
non-CAI groups would be formed ^y random assignment of students. 
When complete random assignment is not possible, it still may be 
possible to have partial random- ass ignment . For example, if more 
students sign "up for a given CAI course than can be accommodated, 
some students obviously would have to be assigned to non-CAI ^courses . 
Rather than our using the typical f i rs t-^come-f irst -served^app roach 
there would be considerable ad\7antage to using a random procedure 
to determine which students will be in the CAI group. 

In addition, it may be possible in some instances to identify 
reasonable groups outside the" institution . For example, if MITRE 
uses one of the Northern Virginia Community Collq^ge District campuses 
as a demons^tration site,, another district campus might serve as a 
reasonable comparison group. 

The comparability of the student characteristics is one of the 
concerns in the formation o.f control groups. Comparability of course 
content is a second concern. Participating-^ institutions will make 
some judgments of course comparability when the determinations of 
6our8e credits are made. Greater comparability could be achieved, 
however, if common instructional materials were used in the CAI and 
non-CAI courses. If possible,, we plan to provide cooperating instruc- 
tors of non-CAI course's with the CAI instructional materials.. These 
instructors would then be asked to use tne CAI materials as extensively 
as possible in their non-CAI - courses . 

Assume we obtain a "matched" sample of students from each of the 
three control groups and they are exposed co "equivalent" courses. 
If the comparison of CAI to non-CAI were similar for each of these 
three control groups, we could be quite confident in our estimate of 
the effect. of CAI, since th^e biases in the three cont^rol groups should 
be very different. Of course the problem of obtaining a good estimate 
of the effect of CAI for any comparison is non- trivial . ' 

"^The most appropriate mode of analysis depends upon which model 
and associated assumptions are thought to be most reasonable; For 
example, iT all the variables that affect posttest scores are used 
as matching variables, and exact matches have' been obtained', then 
the usual average difference in matched pairs is the proper estimate 
of the effect of the treatment. If the matching is liot perfect but 
we believe the relat ionship -between posttest and relevant variables 
is basically linear, the analysis of covariance is appropriate. If 
we "feel that the matching variables are the relevant variables ' but are 
imperfectly measured, then unde-r visual normality assumptions we can 
further adjust scores for errors" of measurement. On the other hand, 
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we- may feel that there are'many other important variables besides 
the matching variables. If we knew, in the absence of a treatment, 
that the difference in posttest scores would equal the difference 
in pretext scores, it follows that the galin over time in difference 
scores is the appropriate estimate of the treatment effect. If we 
knew, in the absence of a treatment* effect , that the treatment group 
would re1|ain the same number of standard deviations below the control 
group on the posttest as on the pretest, the gain over time in stand-, 
ard score differences (i.e., difference in treatment-effect correla- 
tions) is appropriate. We shall use all of these methods (as well 
•as others, perhaps) to estimate the effect of compu t-er-based education. 
With each estimate we shall clearly specify the associated assumptions 
and comment on their plausibility. Conclusions will be based on a 
comparison of the estimates' variability and relative plausibility. 

' Obtaining Matched Pairs of Students with and without CAI 

There are two major classes of problems that arise when using 
matched samples in research. The*first is the trade-off between, 
close individual matches and reduced sample size. This traderbff is 
often addressed as the problem of "attrition" and "incomplete" 
matching." Note that this' problem may exist in experimental as 
well as observational work — increasing preclusion in experiments 
by demanding very tight blocking may reduce the number of units 
available to be studied. * ^ 

The second class of problems that arises when usin^g matched 
sampler is generally discussed only ia observational work. Even if,._. 
we assume exact matches on all matching variables used, are treatment 
and control groups really comparable in the sense of estima/ting 
treatment effects? Two phrases often used to describe the associated 
problems are "sele6tion" and "regression to the mean." These topics 
have been the subject of mu.ch literatu^' recently and continue to be 
a source of controversy* For recent literature on these topics, 
see Campbell and Erlebacher (1970a) and Althauser and^ Rubin (1970). 
The extent to which the obtained treatment and control groups will 
be compara^ble (either by matching or other adjustment)' will be the 
subject of dubstant.ial investigation and discussion when we analyze 
the data. The disdus.sion will of necessity re vol ve . around ^hich of 
several competing models is most appropriate f^or the data in hand • 
The problems of selection and regression to the mean exist only "if 
the model being used to analyze the data is incorrect. As indicated 
earli'er, we will* xxse different models to analyze the data and compare 
re^sulting estimates with respect to the reasonableness of the underlying 
assumptions* Also, by'matching participants fri)m several control ^ 
groups we shall have estimates with different sampling biases. 



For the present, let us restrict ourselves to the first class 
of problems, that of* obtaining good matched sp.mples of reasonable 
size. The recent paper by AJ.thauser and Rubin (1970) fairly care- 
fully describes the relevant .issues, and for further details the 
reader should refer to that article. 

Ideally, for each CAI student we should have a non-GAI. student 
identical in all televant aspects. Clearly this is impossible. First, 
we must restrict our attention to a subset of possibly relevant 
matching variables. It is our judgment that these variables are 
college-file data. / ' ' 

• , 

Next, we must consider sample-size constraints. Clearly, in 
order to choose a matched subset of students we must have a iarger 
sample of non-CAI ' student s than CAI students. Some indications of 
sample size may be found in Rubin (1973a, b). 

The last problem with respect to choosing samples is defining 
a "close" or "best" match. the Althauser and Rubin paper (1970) is 
one of a few giving a complete specification of a'mult ivariate mat'ching 
method. Even though there are no statistical properties derived or 
displayed, t,he method is distinctly a possible one to use. Another 
g,enerally applicable class of methods is presented^ in Rubin (1920). 
Those methods are the multivaria.e generalizations* of univariate 
matching methods investigated in substantial detail (Hubin, 1973a, b) 
and are not only very easy to use in practice but also have some 
very pleasing statistical properties. Current research on these 
methods should enable a more knowledgeable decision to be made shortly 
on choice of matching method. In any case, a reasonable procedure 
may be to produce several metchings by alternative methods' and choose 
the one that satisfied the most criteria. One would hope, by this 
attack, that the resultant matched samples would be as well matched 
as possible, given the constraints on sample size. 

Analyses to Assess- CAI Prb^am Effete tiveness 

Standard covariate ana.lyses . The basic procedures will use 
standard covariance analysis with the pretests, when available, and 
various student" and * school characteristics as the inputs and post- 
test as the output. The basic comparisons will be betwnen the CAI 
students and the mat ched* sets of non-CAI students. 

It should be emphasized, howjever, that due to the quasi- 
e;:perimep*-al nature of the study, such analyses may be more properly 
regarded Jiypothesis exploration than as hypothesis testing. In 
particular, conclusions regarding the relative effects of CAI and 
non-CAI may be weakened by the constraints of the quasi-experimental 
design. 



Covarlance analysis adjusted for errors of measurement s The 
second major approach to. the analyses of program effects will be to 
use adjusted scoreg In^ a uanner similar to the analysis of covar^* 
iance. Although analysis of covariance has been commonly suggested 
in a number of educational and psychological statistics texts as 
an appropriate means of adjusting for pre-existing group difference, 
the technl/que has recently been severely criticized* A number .of ' 
aut'hors have underscored the limitations of this method for the 
purpose of adjusting for pre-existing group differences (e.g., 
CamRbell and Erlebacher, 1970; Evans and An^e^a^io, 1968;''Lord, 196?, 
1969). 'Part of. the difficulty is due to the violation of the assump- 
tion of the equality of within- and between-group regressions. 
Another difficulty is the bias due to unrel-* ability *of the covariate 
(Lord, 1960; Porter, 1967 ; Werts and Lln^x, 1971); 

The data analyses will follow the same lines as the analyses 
described in the preceding section excepts that errors of measurement 
will be taken into account'. 

Ajialy sis of difference scores . The simple difference between 
pretest and porittest is an intuitively natural approach to comparing 
the elementary school CAI students with- non-CAI students. The reason- 
ing behind this approach is simply that in the absence of any treat- 
ment effect one would expect the students ih the two^ groups to grow 
at the same rate on the average. Although rhis simple approach is 
intuitively appealing it has frequently been feulted (see, for example, 
Lord, 1963; and Cronbachand Furby, 1970). The regression effect is 
of particular concern in cases where subjects are not assigned to 
groups at random and there are preexisting differences between the 
groups . 

^ Despite the difficulties with the difference scor$ approach, it 
does p^rovide an alternative form of analysis based on a reasonable ' 
alternative model,. Thus, the analysis o.f difference score will 
provide results which can be compared to those of other analyses and 
enable a comparison of the effects that different asa^pmptions have 
on the results. 

Treatment-effect co rrelations . The four.th major approach to the 
analysis of the effects of programs on the elementary school student 
performance is one recently suggested by Campbell (1970) and referred 
to as treatment-ef fett correlations. In this approach correlation3 
of the posttests with the treatment are compared to correlations of 
the pretest with the treatment. >The treatment varTable is coded 0 or 1. 

^ In situations jwhere the group with lower scores on the f>osttest 
receives the treatment coded 1 and the initially higher scoring group^ 
receives vhe treatment coded 0, a negative point-biserial correlation 
between treatmen^t and pretest will be- obtained. If the treatment is 
effective i^n reducing the between-group variance relative to the 
within-group variance then the correlat ion between treatment and post- 
test would be expected to be closer to zero than the treatment- 
pretest correlation^ 



This analysis is equivalent to an analysis of the difference 
between standaraized pretest aad standardized posttest scores. In 
situations where no treatment effect is expected, initially low- 
scoring groups tend to fall further and further below the norm with 
the passage of time. Fjrom this observation, one would expect that, 
in the absence of a treatment effect, the differences between group 
means of the postt^tft would be larger than differences between group 
means on the pretestp The use of standardized scores or "treatment- 
effect correlations" ia an attempt to adjust for the spread of group 
means in the absence of a treatment effect. 
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TECHNICAL ANALYSIS 




At this point in the p^roject neither the MITRE nor the Illinois 
system is., close enough to full operation to permit, any meaningful - 
measures of technical, performance. Consequently, our interim role 
,wfll be to monitor system development, to comment on decisions that 
will importantly determine the nature of the final systems,, and to' 
see to it that, an adequate method of cdllecting system performance 
data is incorporated into the two projects well before field trials 
begin. In ordej: to monitor developments, a number of ETS staff and . 
two outside consultants visited both Illinois and MITFE in the past 
several months for detailed briefings on the projected hardware and 
system design. Our thoughts on , the, progress to'^date of the two systems 
can be .summarized as follows: 

The TIC C IT System ^ * . . 

It. appears to as that MITRE has made very substantial progress 
over the past three months in the design of the final system. Soft- 
ware .designs appear. to have firmed up considerably, although it still 
remains to be seen whether enough code can he kept core-resident and/ " 
-or wb^i;4heis<,^^ can be optimized enough to support as many ^ 

/^00_J;.e^iSals A The change to an MOS shift , register storage for 
tye video refresh/memory seem^ to us a great improvement over the 
previous 3,phem^.^->although we have some reservations about the new 
method as well. Our principal concern here- is that the dynamic shift 
register memory is especially prone to data Ipss from momentary power 
losses. 'Because TICCIT is expected to run in the average school, 
the system power supplies will have to be designed to handle the 
^substantial voltage fluctuations and occasional monentary power out- 
ages encountered in some communities^ this could prove expensive. 
Orie^ would not expect to provide a full stand-by power sourca .for 
ear^^h. TICCIT installat ion , - but .at^ the very leaat the system ought to 
be capable of recovering auttjmat ically and g*--':efully from momentaiy 
power failures, , without operator interventiou and with litf^e or no 
disruption of student displays. 

The addition of color to the video display increases a concern 
we have already expressed namely, that studf>nts' not receiv?: an 
excessive amount ^of radiation from prolonged exposure to the video ^ 
screen at relatively short viewing distances. Color displays typically 
require higher accelerant ing voltages and generate correspondingly 
more radiation at the tube face. Wt expect to n^ake radiation measure- 
ments ourselves shortly, and would encourage MITRE as well to monxtor 
this aspect of the project. * . • 



The /PLATO System 




PLATO IV appears to be developing more or less on schedule. 
There have b-feen no 'major technical changes i^ the past few months' 
that require comment from us. Our impressic s that the new terminal 
is operating pretty much as expected, and alchough the packaging 
could stand to be improved, the. basic terminal appears to be reliable^ 
Some development work xemains-|or the audio device, but we^ foresee 
no substantial difficulties there. 

/ ' ^ . . 

Two questions that have come up in recent visits deserve some, 
comment. One has to do with the arbitrary limit of 25 ms . of pro- 
cessing time for abuser time si4.ce. In an evening of exploring 
programs under development for PLATO IV, we found thi^s time unit ' 
apparently exceeded a number of times by portions of seyeral differ- 
ent programs. It^ would ap^pear tha-t^at the very least . authors need 
more instruction in how to avotd this problem. The second question 
has/to do with the capacity of the extended core storage. It is 
clear that PLATO cannot in its present ^form support 4000 terminal's 
engaged in 4000 different programs. The system design is clearly 
^uilt around " the assumption that a substantial number of users are 
sharing the same programs (i.e», an entire class is sharing one or 
cit most a few units in ECS), It will be important to obtain fairly 
soon an estimate of just how many different units of "average" size 
(whatever that means) can be simultaneously resident in ECS. 

Data Collection 

We assume that both TICCIT and PLATO will run more or less on 
schedule. Our task, then, will be to assess how well they run . ( ' 
Some of the measurements we expect to make are straightforward, and 
many of these can be drawn from the project's own records. These 
include: 

- Distribution of time to total system failure 

* 

- Analysis of causes of total system failure 

- Distribu't ion of system repair times 

- Analysis of sys^tem repair costs, including labor ai)d parts 

- Estimates of cost of required spare parts and stand-by 
service personnel to keep system operating satisfactorily 

- Distribution of time to terminal failure (including failures 
due to communications lines) " 
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- Analysis of causes of terminal failure 

- Distribution of terminal repair times (including service 
personnel travel time, if applicable) ^ 

- Analysis of terminal repair costs, including labor, travel, 
and parts 

- Estimates of cost of spare parts and/or backup terminals 
required on site to maintain satisfactory system operation 

0 

In addition to these measures of systein reliability, we expect 
to. measure a number of system performance variables, including: 

- Distribution of terminal response times under differing 
system loads 

- Ability of system to withstand and/or recover gracefully from 
power fluctuations or failures, and from communications 
problems 

- Ability of single terminal to recover gracefully from program, 
terminal hardware, or communications problems 



- Analysis of the loads and conditions most likely to seriously 
degrade each system . > 



- Analysis of the relative ease with which system software and 
courseware can be updated 

- Estimates of the magnitude of consulting manpower required to 
help users obtain satisfactory usq of the system 



One special set of measures will have to do with aspects af the 
terminal itself, and vill include: ^ ' 

- Analysis of- terminal* comfort (freedom, from eyestrain, leg- 
ibility of text and pictures, sound levels, keyboard and 

> function "feel") 

- Analysis of terminal safety (radiation hazards, electrical 
hazards, mechanical dangers) 

- Analysis of terminal durability 

- Special problems with terminals in schools (special electrical 
or environmental requirements, terminal size, special wiring 
requirements) 
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The pro'^ject reporter has compiled a bibliography of recent 
writings, both -prof essional and journalistic, related to the subject 
of computer-basfed education in general and to the specific technical 
concerns of our research, development, and evaluation. Since the 
work in progiess is complex and has broad ramifications, writings 
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* * * * 
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3^ Related Educational Issues \ 

* Ar tides of opinion on such educational' concerns as the indivi- 
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education"), and the extension of "humanistic*' education. The 
computer is seen as having an effect on new a^^d perennial issues in 
education* 
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6^ Evaluatio n * * . 
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7> .Development of Instruc tional/ Kaxterlals 

This section includes general, theoretical articles on how 
curriculum should be developed for CAT ,and what kinds of organizations 
would best serve curriculum development (i.e., centralized or decen- 
tralized?). It also contains discussions of specific curriculums 
developed. for computer-based ^nd computer-assisted instruction. ^' ' 

, . - ■ ^ \ 
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Mitzel, H. E. Computers in Inst . action — preparation of instruction 
.materials by nonprdfit consortia. Paper delivered at Rund 
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that have been implemented in an elementary school tea,ching. 
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Alexandria, Va.: HumRRO Professional Paper 20-71, October, 1971. 
A specialized organization needs to be the focal point for 
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SURVEY OF INSTRUCTOR ACTIVITIES AND ATTITUDES 



As you know, your school is one of several community colleges that 
has agreed to participate in one of two 1' -ge scale demonstrations of 
computer-assisted instruction (CAI) , funded by the National Science 
Foundation. These demonstrations will attempt to show that CAI 

f 

is technically sound, economically feasible, and educationally effective. 
The Education^il Testing Service of Princeton, New Jersey will monitor 
these demonstrations and evaluate their technical, economic, and 
educational effectiveness. We are conducting this evaluation undefr a 
separate contract with the National Science Foundation so that we are 
not dependent upon the organizations that are operating these CAI 
systems but are responsible only to NSF and the public to insure that a 
thorough arid objective evaluation is conducted. 

Wliile the educational evaluation will focus mainly upon student 
achievement and attitudes, we are also interested in the impact of 
CAI on your daily activities and your attitudes toward the various 
aspects of this project. Therefore, ne^r the end of each term, we will 
be asking a few faculty members from your^ college to get together in a 
^ small group in order to discuss issues and opinions which have arisen 
in regard to CAI. These discussions will be held without the presence 
of any administrators or those responsible for the project, and we will 
not identify, in any way, the individuals making conanents or giving us 
opinions. Regardless of whether you are authoring materials, planning 



to use materials, unfamiliar with the project, or opposed jto the project, 
we. hope that you will be willing to share your opinions with us. If 
you are willing to participate in a 1-2 hour discussion on this topic, 
please indicate this on the first page of the questionnaire. Your 
affirmative response is merely an expression of interest and not a firm 
commitment. 

We are also asking you and your colleagues, including all faculty 
members, counselors, and other professional staff members, to respond 
to a series of questionnaires; We apologize in advance fdr any 
'inconvenience this may cause you and wish to assure you that the number 
of questionnaires will be kept to an absolute minimum, certainly no 
more than two per year, and that their length will never be greater than 
this first one. We hope that you will agree with us that the potential 
implications of these CAI demonstrations and the Veed for thorough 
evaluation justify the time needed to complete this questionnaire. 

This first questionnaire has been designed to sui^vey your initial 
impressions and existing opinions on a variety of topics related to 
CAI before you get very deeply involved in the project. Your complete 
candor in answering these questions will be greatly appreciated. 
Needless to say, we will hold your individual responses in strictest 
confidence. However, you may also be assured that the summary results 
will-be shared with the CAI project directors so that they may take 
the group's opinions into account while implemenjting these demonstrations. 

The first section consists of questions about your background, 
present status, and teaching practices. The second section asks for 
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your opinions regarding computers and CAI as you now understand it. 

The final section focuses on your opinions regarding various educational 

practices which may be influenced by these projects. Aftet: completing 

the questions and adding any comments about the project or thq questionnaire, 

please return the questionnaire in the attached pre-addressed, business 

reply envelope. 

m 



f 



NAME 



COLLEGE 
DIVISION 
DEPARTMENT 
POSITION 



Please oircle che appropriate letter. 



1* Are you willing to participate with other faculty members and a 

representative of ETS in a confidential discussion of this project? 

a. Yes 

b. No . 

2. Are you the chainnan of your department or division? 

Department chairman 

b. Division chairman 

c. Both Ojl the above 

d. None of the above 



3. Do you have another other administrative duties, whtther or not you 
a department or division chairman? 

a. Full-time (specify) 

b. Part-time (specify; 

c. None ' j 



Is your position (other than chairmanship or temporary administrative 
assignment) an adjunct, acting, or temporary (less than 9 months) 
position? ' 



a. Adjunct 

b. Acting 

c. Temporary 

d. All three of the above 



e • Ad j unct and ac t ing 

f. Adjunct and temporary 

g. Acting and temporary 

h. None of the above 
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Please indicate the number of years of 
September 1972 you have in each of the 


working experience prior to 
following categories. 








Number of 


Years 






Less 
Than 1 




3-5 


6-10 


More 
Than 


£*j.enien Ldry scnoo J. Leacnxng 


A 


B 


C 


D • 


£* 


Secondary school teaching 


A 


B 


C 


D 


E 


Commuulty co'^lege teaching 


A 


B 


C 


D 


E 


College/university teaching 


A 


B 


C 


D 


E 


Educational research or administration 


A 


B 


C 


.D 


E 


Non-teaching experience which is 
directly related to your courses 


A 


B 


C 


D 


E 


Non-related working experience 


A 


B 


C 


D 


E 



6. Please indicate the courses and the number of sections of each that 
you are teaching this term. 



Number of Sections 



//I 
#2 
//3 

//5 



ERIC 



Questions 7 to 23 concern the courses you listed in question 6. Column 
#1 refers to the 1st course you listed, //2 to the second, and so forth. 
Please circle one alternative per colum.a using the same n'omber of columns 
as courses listed in question 6. In other words, circle one letter in the 
first column to Indicate your response in relationship, to course //I, one 
letter in the second column for course ^2, and so fo^th. 

7. Who selected the textbook(s) to be ised In each class? 

(Courses as listed in Question j>) 





//I 


//2 


//3 


//4 


//5 


Division ch.'^lrman 


A 


A 


A 


A 


A 


Department, chairman 


B 


B 

1 


B 


B 


B 


^ Academic dean 


C 


" C 


C 


C 


C 


Department committee 


D 


-D 


D 


D 


D 


Personal decision of instructor 


E 


' E 


E 


E 


E 


Other (specify) 


F 


F 


F 


F 


F 
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8. What factor most heavily influences the content of each course? 

(Courses as listed in Question 6) 





111 








111 


Personal decision of instructor 


A 


A 


A 


A 


A 


Committee o^f department members 


B 


B 


B 


* B 


B 


Dean or department chairman 


C 


c- 


C 


. C 


C 


Recommendations of professional 
organizations 


D 


D 


D 


D 


D 


State department iSf education 


E 


E 


E- 


E 


E 


Professional experts from an outside 
-s^ ^ organization 


F 


F 


F 


F 


F 


' Students 


G 


G 


G 


G 


G 


Other (specify) 


H 


H 


H 


H 


H 



9. Which is the second most important factor in determining course content? 

(Courses as listed in Question 6) 





11 


111 


il 


i± 


111 


There is only one factor. 


A 


A 


A 


A 


' A 


Personal decision of instructor 


b' 


B 


B 


B 


B 


Committee of department members 


C 


C 


C 


C 


C 


Dean or department chairman 


D 


D 


D 


D 


D 


Recommendations of professional 
* organisations 


E 


E 


E 


E 


• 

E' 


State department of education 


F 


F 


? 


F 


F 


Professional experts from an outside 

organization 


G 


G 


G 


G 


G 


Students 


H 


H 


H 


H 


K 


Other (specify) 


I 


I 


I 


r 


I 



10* Which of the following is most important for students to learn in 
each coarse? 

(Courses as listed in Question 6) 



ERIC 





ik 


111 


il 


H 


il 


Specific skills 


A 


A 


A 


A 


A 


Theories > concepts, general understanding 


B 


B ' 


B 


B 


B 


^ Specific facts or a body of knowledge 

Values, appreciations, attitudes towatd 

subject 


C 


C 


C 


C 


C 


D 


D 


D 


D 


D 


Sense of personal progress, self- 
^ * confidence or esteem 


e 


E 


E 


• 

E 


E 


Other (specify) 


F 


F 


F 


F 


F 



11. Which is the second most important for students to learn? 

(Courses as listed in Question 6) 

n il 113 H is 

Specific skills A A A A A 

Theories, concepts, general understanding B B B B B 

Specific facts or a body of knowledge C C C C C 

Values, appreciations, attitudes toward 

/ subject D D . D D D 

Sense of personal progress, self- 
confidence or esteem E E E E E 

Other (specify) . F ^ F F F F 

12, If the objectives of the course are stated to students, how are 
these objectives stated with the most emphasis? 

(Courses as listed in Question 6) 





n 


il' 


il 


H 


il 


in terms of the content to be covered 


A 


A 


A 


A 


A 


In terms of the requirements to pass 
or attain particular grades 


B 


B 


B 


B 


B 


In terms of desired behaviors or 
abilities at thp end of the course 


C 


C 


C 


C 


C 


Not stated specifically 


D 


D 


D 


D 


D 


Other (specify) 


E 


■E 


E 


^E 


E 



13, If the objectives of the course are stated to students, what is 
th6 first way this is done? 

(Cdurses as listed in Question 6) 





ii 


12 


ill 


H 


,£5 


In a publication available to all 
students prior to registration 


A 


A 


A 


A 


A 


In a Jiand-out at the beginning 
of the lourse 


B 


B 


B 


B 


B 


Verbally during the first few 
class periods 


C 


C 


C 


C 


C 


During the course as an integral part of 

- instruction 


D 


D 


D 


D 


D 


Not stated specifically 


E 


E 


E 


E 


E 


Other (specify) 


F 


F 


F 


F 


F 
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14* Which of the following is the most important source of test 
questions for you? 

(Courses listed in Question 6) 





ill 


ill 


111 


H 






Texcbook publisher 


A 


A 


A 




A 


> 


Department coiiunittee 


B 


B 


B 


B 


B 




Personal effort 


C 


C 


C 


C 


C 




^ Colleagues (past or present) 


b 


D 


D 


D 


D 




Standardized tests 


£ 




£ 


E 


£ 




Other (specify) 


F 


F 


F 


F 


F 




No tests are given. 


G 


G 


G 


G 


G 





15. Which is your second most important source, of test questions? 

(Courses listed in Question 6)' 





n 


n 


£3 


#4 


111 


Textbook publisher 


A 


A 


A 


A 


A 


Department committee 


B 


B 


B 


B 


B 


Personal effort 


C 


C 


C 


C 


C 


Colleagues (past or present) 


D 


D 


D 


D 


D 


Standardized tests 


£ 


£ 


£ 


£ 


£ 


specify) 


F 


F 


F 


F 


F 


No tests are given. 


G 


G 


G 




G 



16. What type of test questions do you use most often? 

(Courses listed in Question 6) 





n 


HI 


HI 




HI 


Multiple-choice questions 


A 


. A 


A 


A 


A 


Matching questions 


B 


B 


B 


B 


B 


Other objective questions 


C 


4 

c 


C 


C 


C 


Essay questions 


D 


D 


D 


D 


D 


Short-answer quesi:ions 


£ 


E 


E 


E 


E 


Fill-in-the-blank questions 


F 


F 


F 


F 


F 


Cspecify) 


G 


G 


G 


G 


G 


No tests are given. 


H 


> H 


H 


H 


H 
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17. * What is the second most conunon type of question in your tests? 

(Courses as listed in Question 6) 

il 111 il //5 

Only one type is used. AAA 
Multiple-choice questions B B B 

Matching questions C 
Other objective questions D 
Essay questions E 
Short-answer questions 
Fill-in-the-blarik questions 
^ Other (specify) 



F 

G 
H' 
I 



C 
D 
E 
F 
G 
H 

T 



C 

D 
E 
F 
G 
H 
I 



A 
B 
C 
D 
E 



F 
G 
H 

r 



A 
B 
C 
D 
E 
F- 
G 
H 
I 



No tests are given. 

18. By whom are the final grades for each course primarily determined? 

(Courses as listed in Question 6) 





il 




111 




il 


Individual instructor 


1 

'A 


A 


. A 


A 


A 


Committee of faculty who teach 
various sections of course 


B 


B 


B 


B 


B 


Course coordinator 


C 


C 


C 


C 


C 


Results of standardized test 


D 


D 


D 


D 


D 


Individual student and instructor 
in consultation 


E 


E 


■ E 


E 


E 


Other (specify) 


F 


F 


F 


F 


F 



• \ 



19. How much of your final course grades are determined by student's 
papers or projects? 

(Courses as listed in Question 6) 





{fl 


//2 


//3 


//4 


//5 


More than half 


A 


A 


A 


A 


A 


Substantial amount 


'B 


B 


B 


B 


B 


Fair amount 


C 


C 


C 


C 


C 


Very little 


D 


D 


D 


D 


D 


None 


E 


E 


E 


E 


E 
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20, How much of your final course grades are determined by weekly 
' or more frequent homework assignments done out of class? ^ 

(Courses as listed in Question 6) 







111 


il 


H 


£5 


More than half 


A 


A 


A 


A 


A 


Substantial amount 


B 


B 


B 


B 


B 


F^ir amount 


C 


C 


C 


C 


C 


Very little 




D 


D 


D 


D 


None 


£ 


E 


£ 


E 


£ 



21* How much. of your final course grades are determined by atj:endance, 
recitation, or lab work in class? ^ 

(Courses as listed in Question 6) 





11 


il 


il 


H 


il 


More than half 


A 


A 


A 


A 


A 


Substantial amount 


B 


B 


B 


B 


B 


Fair amount 


C 


^ C 


C 


C 


C 


Very little 


D 


D 


D 


D 


D 


None 


£ 


•E 


£ 


E 


£ 



22. How much of your final course grades are determined by quizzes, tests, 
or examinations which are completed either in class or out 2)f class? 



^ (Courses as listed in Question 6)^ 

( 



* 


il 


il 


il 


il 


il 


More than half 


A 


A 


A 


A 


A 


Substantial amount 


B 


B 


B 


* B 


B 


Fair amount 


C 


C 


C 


^ C 


C 


Very little 


D 


D 


D 


D 


D 


None 


E 


£ 


£ 


£ 


E 



23. How are final grades in each course distributed? 

9 

(Courses as listed in Question 6) 

£1 £2 ^ M i5 

With set proportions for each ^ 

grade ("on the curve") A A A A A 

According to an ab^.olute standard B ^ B B B B 

According to a relative standard^ which is 

based on overall class performance C C C C C 

Other (specify) D D D D D 



\ 

24. Which of the following instructional techniques or resources have 
you used in any of your courses? (Circle all that apply.) 



Lectures 



b. Discussion S€«;rXions 

> c . Field * expeditions 

'd.^. Laboratory experiences 

e. Individual assignments (e.g, papers, 

class presentations, projects) 

f. Group assignments 

g. Textbooks 

h. Books •c^ readings 

i. Workbooks 

j . Outside readings 



'k. Cassettes 

1. Tapes 

m. t^rograramed texts 

n. Computers 

o. Film strips or slides 

p* Motion pictures 

q . Recorders 

r . Television 

s. Other (specify) 



25. Are there any aaditional instructional resources you would like to use 
^ in your courses? 

a. No 

b. Yes (specify) ^ 



What prevents their use? 
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26» What is the ::-verage time you spend preparing a course before the 
term begins? 

a. Less than one week 

b. More than one week and less than three weeks 
c» More than three week^ and less than five weeks 
d. More than five weeks * 



27 • Wliat is the average time you spend during the term preparing for 
each hour of class time? If you make the same presentat;Lon to 
more than one section, count it a? only one hour of class time. 

a. Less than one hour . 

b. More than one hour and less than two hours 

c. More than two hours and less than three hours 

d. Morft than three hours , _ 



28. 



Of the time you spend dealing with your teaching duties (in- a broad 
sense), approximately how much is spent on each of the following activiti^i? 



N 



More 





Than 


Substantial 


Fair 


Very 






Half 


Amount 


Amount 


Little 


♦None 


Conducting class 
Preparing lectures, 


A 


B 


C 


D 


E 


discussions , etc. 


A 


B 


C 


D 


E 


Preparing tests 
Grading tes'^ts 


A 
A 


B 

B 


C 
C 


D 
D 


E ^ 
E 


Counseling students on 












course work 


A 


B 


C 


b 


E 


Counseling students on 












ether matters 


A 


B 


C 


D 




Advising or coaching student 












organizations 


A 


B 


C 


D 


E 


Informal discussions with 












students 


A 


B 


C 


D 


E 


Committee meetings. 


A 


B 


C 


D 


E 


Administrative duties 


A 


B 


C 


D 


E 


Interaction with colleagues 


A 


B 


C 


D 


E 


Reading or attending lectures 












to enhance your own knowledge 


A 


B , 


C 


D 


E 


Other (specify) 


A 


B 


C 


D 


E 



ERIC 



-10- 



29. In which of the following activities have you participated at any 
time? (Circle as many as apply.) j 

a. Taking a course on computer programming. 

h. Working as a computer operator. 

c. Working as a computer programmer* 

(\ Working on systems design problems. 

fc. Writing computer programs for research purposes* 

f. Watching a demonstration of CAI* 

g. Taking a course which used CAI materials. 

h. Teaching -a course which usefd CAI materials* 

i. Writing CAI materials. 

J. Serving on a committee to-^design an introductory course 
involving several sections and/or team teaching* 

k. Taking a course on instructional design* 

1. Serving on a national commission to set standards for courses in 
a particular area* 

u. Writing a textbook* 



n. 



Writing course objectives for dissemination to colleagues or 
students. 

Writing e course outline for dissemination to colleagues or 
students. ^ s 

Other related experiences (specify) 



» / 
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Section II: Computers and Computer-Assisted Instruction 

Please circle the letter of the most appropriate response* 

30. If your charge account bill has an error, it was probably caused by 
a* malfunctioning of the machine. 

b. mistakes in the in:^ormation* given to the computer. 

c. inferior design, or programming, of the ' com^^uter by some person. 
d» inherent shortcomings of machines 

e. factors which are unknown to me. 

f . ' other (specify) 



31. The best use of computer-assisted instruction (CAI) would be 

a. to take over some courses completely. 

b. to take over parts of some courses. 

c. as an alternative for students who prefer it. 

d. as an adjunct to a regular cour/se for students who are interested. 

e. for research, purposes only. ' 

. f. other (specify) I 

♦ ■ — ■ j • — 

32 • The best use of CAI would be with 

a. introductory or lower level courses. 

b. advanced courses in the scienceo, mathematics, or computer science. 

c. courses which-require a great Jeal of memorization. 
' d. other (specify) ^_ 

33. What woi*ld be the best way for CAI to be used in relation fto the 
following parts of instruction? (Circle one letter in each row.) 

Partial Total No 

Supplement Replacement Replacement Use Other (specify ) 



Lectures 


A 


B 


C 


D 


E 


Textbooks 


A 


B , 


C 


D 


E 


Tutoring 


A 


B 


c 


D 


F 


Counseling 


A 


B' 


c 


D 


E 


Discussion groups 


A 


B 


c 


D 


E 


Lab exercises 


A . 


B 


c 


D 


E 


Other (specify) 


A 


B 


c 


D 


E 



student reactions to CAI will probably be 

a. negative no matter how much or how little it is used, 

b. negative only if it is used to take over a course complete 

c. varied and dependent upon the, student's background. 

d. dependent upon the quality of the material. 

e. positive if it is used as an adjunct to regular courses. 

f . positive no matter how much or how little it is used. 

g. other (specify) ' 



The best criterion- for judging the success of CAI would be 

a. ^student achievement. 

b. student attitudes. 

c. faculty acceptance. 

d. financial considerations. 

e. other (specify) , 

The primary advantage of CAI is that it may eventually 

a. relieve the instructor of routine duties and give him more 

time for teaching. 

b. save money. 

c. be a better means of instruction for some students^ 

d. provide remedial instruction for those who need it. 

e. other (specify) 



I 

CO 



C 
0) 

B 

0) 
u 
<T3 
u 
W 

c 
o 



o 

U-l 
0) 

u-i 
o 

c 
o 

c 



O 
2 



0) 
0) 

Mi 
< 



to 

C 0) 

O Q) 

U U 

C/3 < 



PQ 



a 




















o 




es 
















u 


U 


w 














\ 




•H 


o 












fH 




o 




a 


rH 










03 






Q) 


u 


0) 










0) 






C 




iJ 














0) 


Q) 


a 


0) 






ij 








iJ 






iH 










O 


CO 


tTJ 




0) 


a 






c 




c 




O 


rH 


a 


e 


c 




•H 






c 


•H 


rH 


o 


o 


•H 








CO 


0) 




•H 


CO 


o 










to 
















c 




x: 








u 


CO 


c 




CO 












o 


u 










/-^ 


CO 


O 


a 




o 


;J 




u 


ij 


n 






•H 








0) 


o 


0) 


<: 


0) 






fH 




O 


»H 


a 


•H 


O 


60 




•i. 




fH 


a 


O 


6 


O 




0) 


0) 




U-{ 




B 


O 


»H 


O 




fH 


iJ 


rH 


0) 


CO 




CO 




CO 


c 


rH 


U 


O 


W 








0) 




o 


o 


0) 


O 






0) 


u 






•H 


o 


fH 












o 




U 








0) 


6 


o 


o 


6 


o 


O 




0) 


Q) 






B 








O 


o 


iJ 


rH 




0) 






0) 




u 




<\J 






o 


to 


o 


B 


o 


u 


c 


•H 




CO 




C 




O 




CO 


o 


>-« 






fH 


•H 


to 


o 


00 




•H 


a 


rH 


0) 


a 


S 


c 




c 


•H 


4J 


o 


CO 




0) 


o 










CO 


u 


> 




u 


o 


c 




c 




o 


a 




a 




0) 


o 




o 


0) 




a 




6 


u 






fH 


•H 


u 








o 






u 






CO 


0) 




Q) 


o 


> 




o 


O 


o 


•H 














C ' 




c 


CO 


0) 




« 




c 


CO 




CO 




CO 


X 




W 




fH 


CO 


CM 


CO 




CO 

1 


iJ 




W . 


O 






>^ 


u 




u 


c 




Q) 




»H 


0) 


CO 


0) 


CO 


0) 


•H 


O 


AJ C 


ij 


:^ 






iJ 










:3 -H 














0) 


•H 


a ^ 


c 


>^ 


a 


u 


a 




a 


u 


O 


6 c 




Q) 


6 


0) 


6 


0) 


6 


CO 




O <TJ 


> 




o 


> 


o 


> 


o 


iH ^ 


(1) 


o s 


w 




o 


0) 




0) 


u 


a 


w 








































0) 


• 


















iH 




00 




ON 




d 




fH 




P4 


CO 


CO 

















Q Q Q a a a o 



o o o a o a a 



PQ PQ PQ PQ PQ PQ PQ 



CO 
Q) 
CO 

u 

o 

a 

X 
u 



to JC 
C *J 
•H -H 



'V 



CO 

<1> 



c 

- o 

0) 
(0 
X 

J 



60 

c 

O 

to 

0) 



a cj 



CQ PQ 



CO 

u 

to o 

C u 
•H O 

X a 

O P 
to -u 
0) CO 

u C 

to c 

6 to 







rH 


•H 


^3 


•H 


fH 








rH 


4J 


C 


X 




0) CO 










*H 




to 


X u 






U^ 


> 






u 








CO 








•H 


0) 






CO 




C 




CO 


u-l 






0) 


o 


to 


o 


OJ 


O 0 






o 


to 




•H 


^3 


u 






o 






iJ 




0) CO 




rH 




0) 


o 


O 


>^ 


c 


X) 


to 


CO 


> 


to 




fH 


o o 


to 


c 










X 


iJ 


u^ 


o 








iJ 


00 


0) 




CO 


0) 


to 


0) 


CO 


•H 


^ 0) 


60 


u 


CO 


0) 


0) 


c 


X 


fH 




0) 








•H 






'H 


a 




o 






to 


fH to 


CO 


S 


0) 




0) 






r: fH 


CO 


*H 








o 


iJ 


o -H 


to 






C 


to 




•H 


to 


a 


O 


iJ 




•H 


(0 




^3 > 




O 


o 






0) 


0) 


fH to 


to 


u 


c 


0) 


0) 


CO 










c 








to 


o CO 


CO 


CO 


to 








Fi 


jc -v 


*H 


'H 


o 


X 


•H 


o 


c« o 








o 




o 






H 


»H 


n 


■H 


0) 




n 


M AJ 


< 


< 


< 




X 


u-l 


< 


< 0) 


O 


O 






H 


o 




O B 






<r 










\o 



















Rir 



I 



U 
CO 



o 



0) 



00 

C 0) 
O <D 

C/) <J 



PQ 



W W W W W W 



Q Q Q Q Q Q 



U U U U U O 



PQ 



PQ 



PQ 



PQ 



PQ 



PQ 



W 



PQ 



0) 60 
•H 

































0) 




D 


CO 




o 


iH 




























a 




0 


a 




•H 




M 












CO 














a 












O 


<D 












iH 














o 






0 




sn 


x: 
*j 










■(J 


03 
•H 






o 




M 

<: 




rH 

0) 




c 
0 


0 




0) 










C 








o 








u 














4J 


CO 


M 




iH 


o 




<D 


0) 


c 




CO 






CO 












c 


tH 














4J 


03 








p. 


0) 






03 


03 




<u 


(0 


CO 






(0 






03 


'4-* 




o 




o 


CO 








C 




c 


•H 


u 


V-i 




u 




iH 


6 










iH 


u 






D 


•H 




o 




c 


<D 


(0 


c 




iH 




03 








0) 




c 




'V 


a 






0) 


0) 












<u 










> 


o 


03 




0) 


•H 




!• 






c 








:5 


CO 


C 








<D 


o 


o 






iH 




o 




u 


0) 


QJ 


D 






u 


03 




■(J 




•o 








c 


0) 




o 


S 


u 


o 


U 


u 






D 


x: 










CO 


















C 


(0 




< 


O 


■u 




O 








•H 






a 








a 


0) 


QJ 








o 






u 




iH 




C 




CO 






c 


« 


0) 


o 


O 


00 




















00 




u 
















c 
















O 


c 


•H 




<u 


c 






D 






0) 


•rl 












03 




O 




CO 




4J 


03 






O 


a 


O 


o 


iH 








a 




0) 






B 








00 






o 


•H 


CO 




3 






03 
















a 




> 




Q 


<u 


3 








■(J 


00 




x: 




o 


0) 


CO 




a 


C 






M 


Qi 




o 




e 


Cu 


c 




■u 




■u 




o 




0 


•H 




^ XI 


<: 


'V 




(0 








0) 


■H 








o 


■u 


B 




0 


4J 






u 


03 




<i> 


o 




o 






iH 














CO 








O 


Q) 




CO 


CO 




o 


■u 




03 






<D 


<D 


03 


tM 


<D 








03 


N 






iH 


O 


03 


o 


<u 








CO 




'O 




u 










•H 


-n 




03 


u 




c 


CI 


■U 


CO 


CO 




■U 


0 










> 




QJ 


o 


C 


a 






•H 


vC 


u 


c 








0) 


0 




c 


•H 




03 


^3 




•H 




O 






0) 


o 


•H 








CO 


•H 










U 


•H 






■U 






o 


4-J 






C3 




c 




U 




(0 4^ 


o 


'V 


4J 


t-< 






CO 






03 


O 


S 


0) 


03 




0 




CO 


CO 




0) 


0) 


c 


03 


»-< 


0) 


0) 


c 




B 


Cu 


0) 






<D 




0) 


0 


iJ 




tM 












■u 


0) 


0) 






B 




CO 


C 




iH 


C 


c 


O X 


tM 


03 


o 


■u 


•H 




03 


B 




<^ 


03 




«) 


iH 


0 


03 


Xi 


00 


0) 


o 


0) 


a 




C 


CO 


M 


0) 


a 




C 






a 


03 






•H 


03 










03 


0) 


c 


< 




o 


CO 


•H 


CO 




03 


•H 


CO 




CO 


•H 




o 


0) 






o 


o 


U 


00 iH 


•H 








O 




V-! 




CO 






P c 






C 




a 






0) 




o 








0) 




0) 


0 




CO 


0) c 






03 


0) 


CO 


0) 


0) 


> 


w 


03 


M 


O 




M 




a 


0) 




> CT3 


o 


0) 




x: 


0) 






0) 




0) 


<: 


03 




03 


<: 


c 








w o 


H 




4J 


H 


u 


■(J 


H 




u 




u 


tM 




S 


0 




< 


u 


0 



00 



o 

in 



in 



in 
vn 



-15- 



Section III;, --Eclucational Practices 

Please circle the letter of the most appropriate response. 

56. The content to be covered in a course should be determined by 

a. each instructor individually. 

b. a committee of department members. 

c. a dean or department chairman 

d. the recommendations of professional organizations. 

e. the state department of education. 

f. professional experts from an outside organization. 

g. students. 

h. other (specify) 



57. Students should have some influence on 

a. the topics covered in a course. 

b. the way topics are presented to the class. 

c. Soth of the above. 

d. their own assignments only. 

e. none of the above. 

f. other (specify) 



58. The primary basis for the organization of a course should be 

a. the intrinsic organization of the subject matter. 

b. student interests. 

c. the organization of the textbook. 

d. the preferences of the instructor. 

e. other (specify) 



59. Individualized material beyond class presentations should be given 

a. only to students who cannot handle the regular work. 

b. only to exceptionally bright students. 

c. to both of the above. 

d. to all students. 

e. other (specify) ; 
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60. Final grades in a course should be determined by 
a. the individual instructor. 

b*^a committee of faculty teaching sections of that course, or 
the individual instructor if there is only one section. 

c. the department chairman or academic dean. 

d. a committee of faculty who are not directly involved in 

teaching the course. 

e. an external examination or evaluator. 

f. other (specify) ^ 



61. Because of the ways in which grades are presently used, they are 

a. a positive incentive that keeps many students working and learning. 

b. a threat that inhibits students and cause them much anxiety • 

c. a detriment to education. * 

d. a necessary threat that pre3er\res the discipline needed for 

learning to occur. 

e. other (specify) 



63. The results of tests should be used 

a. to determine a student's final grade. 

b. , to diagno.se a student's strengths and weaknesses. 

c. to evaluate the instructional methods being used. 

d. (a) and (b) 

e. (a) and (c) 

f. (b) and (c) 

g. all of the above 

h. other (specify) 
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C.R.P.T. 

ATTITUDES TOWARD Rl'ADING 
GRADE 2 



O . 



STUDENT'S NAME 



.1 

O 
QO 



A. I am ea+ln 
Candy. 



I AM VERY I AH A LITTLE I AM A LITTLE j a'I V^p^ 
SAD BITSAU ^BITIiAPPY MFPi 




I AM VERY I AM A LITTLE I AM^ LITTLE i AM VFRY 

B. I hove a sVomocK bit happy H/.ppf 

. ocVie. 



C. I IdsV my \uncK 




I AM VERY I AM A LITTLE I AM A LITTIil t AM V^K- 
SAD BIT S/.D BIT fWPPY j^^pp.^ 



box "tociciy. 




visif The Xoo. 



I AM VERY I AM A LITTLE I AM A LITTL 
SAD BIT SAD BIT iiAPPY 



I AM'VLUY 




Id 

ERIC 



Pcige i. 



I. I am \»ornin 



to rMd. 



13 



I AM VERY I AM A LITTLE I AM A LITTLE t AM V'^J'v 
SAD BIT SAU BIT liAPPY "jipp/"^ 




2.Tcxlay our redding 
Class wcis cancellec). 



I AM VERY I AM A LITTLE I AM A LITTLE j a.,< v=-Hv 
SAD BIT SAD BIT IIAPPY h/.VpY ^ 




3. Someone gave me o 

book for my 
bir-VKdUiy, 



I AM VERY I AM A LITTLE: I AM A LITTLE t AM V^^Y 
SAD BIT SAD BIT IL\PPY H.\PPr 




't. I am \«sWn\ng 
i&ocVw^r read a sVory. 



5.SorYve<M\e ^ook 
library book dway 
-from W. ' 



4. I cm> reading ^ "VW 
WKo\e ckiss. 



ERIC 



I AM VERY I AM A LITTLE: I AM A LITTIX -i AM VF^Y 
SAD BIT SAD BIT lli\PPY h/iPPy' ' 




I AM VERY I AM A LITTLE I AM A LITTLE j am VEP.^ 
SAD BIT SAD BIT flAPPY HAPPY " 




I AM VERY I AM A LITTLE I AM A LITTLE 
SAD BIT SAD BIT IIAPPY 



I AM'VLRY 
Hv\PPY 




« 



I 



Pagt Z , 



-rr- 



I f ocAc a book \o 
bed \^UVi me \qsV° 



I AM VERY I AM A LITTIJi I AM A LITTLE i AM -VERy' 
SAD BIT SAD BIT HAPPY haPPY 




&.| am ck i^rribW 



I AM VERY I AM A LITTLE I AM A LITTLE i AM VERY 
SAD BIT SAD BIT IIAPPY MPPY" 





library. 



I AM VERY I AM A LITTLE I AM A LITTLE j AM VERY 
SAD BIT SAD BIT IIAPPY Hi\Fpi: ' 




ID. I ju5^ learneil some 
new wor3s . 



I AM VERY I AM A LITTLE I AM A LITTLE i AM VERY 
SAD BIT SAD- BIT HAPPY H/.FPY 





II. \ lost- my neodin^ 
bcxk Vocioy . 



I AM VERY I AM A LITTLE I AM A LITTIi: j am V7' ' 
SAD BIT SAD BIT fL\PPY Hi\PPY 





1 am looWiVi^ CI 
viordi '\t\ tKe. 
dlidrtonary. 



I AM VERY I AM A LITTLE I AM A LF.TL 
SAD BIT SAD BIT }L\PPX 



I AM'VE.--.: 
Hi'vPPi 




ERIC 




Pa^e 3 . 



I AM VERY I AM A LITTLE I AM A LITTLE i AM VERY 
<VD BIT SAD BIT IIAPPY HAPPY 



IB. My Sishcr is 
recsclivi^ me a 



I . , , , . / I AM VERY -I AM A LITTLE I AH A LITTLE i AM M^RY 

\% \ am sitttriA' under ^ad bit sa-d bit {iappy happy 




reading 'Ibdkoy. 




I AM VERY I AM A LITTLE I AM A LITTLE i AM V^RY 
SAD^ . BIT SAD 'BIT IIAPPY HAPPY 




poem. 



I AM VERY I AM A LITTLE I AM A LITTLE i AM VERY 
SAD BIT SAD BIT IIAPPY hAP^Y 




*7. newspaper *s 
4oo hard -for n\c +o 
trad hy myseVf. 



I AM VERY I AM A LITTLE I- AM A LITTLE j AM VERY 
SAD BIT SAD ^ BIT IIAPPY Hi\PPY 




Pacya. 'V; 



7 



\ am supposed 4°o 

Wr'il^e a slory m 
Class tomorrow. 



I AM VERY I AM A LITTLE I AM A LITTUE . i AM .VERY 
SAD . BIT SAU BIT/l'lAPPY- h/vPPY 



2o, I am -H^e slouiesf 
reader in mf c\ass . 




I AM VERY J AM A LITTLE I AM A LITmi 
-^D ' . BIT SAD bit lIAPpf 



I AM VERY 
HAPPY 




Zl. I cat\ wrAe a\\ of 



I AM VERY -I AM A LITTLE I AM A LITTLE 
SAD BIT SAD BIT IIAPPY 



I AM VERY 
H/vFPY 




I AM VERY I AM A LITTLE I AM A LITTLE 
SAD BlT SAU * BIT TbV^PY 



I AM VERY 
H/iPPY 




lW leodher- gave 
nst. a s^ory Vo reodi^ 



I AM VERY I AM A KITTLE I AM / LITTLE 
SAD . J BlT^D BIT IL\PPY 



I A.'r-v'Eh 

MFP: 





2A. I Can read oiV 
of •rif>c s^w^svgns; 



I AM VERY I AM A LITTLE I AM A LI^TI 
SAD BIT SAD BIT ii/vPPY 



AM' VERY 
t{i\?t''{ 



Item No. 599148 ^ 




O 
o 



c.R.p.': 



Attitudes Townrd I'endin/', 



f.rndos A £, 6 



CO 



^ :> 



Road the following statements silently as thev ai'e read aloud to 
yr;i one at a time. Then, if you i^rce with the statement, circle 
the +. If vou disagree, circle the - . If you agree very much or 
stronglv, circl( the -H- . If you disagree very much or strongly, 
circle the = . 



S trongl y 
Agree Agree 



learning to rt'<id is ver^ 
ir^noVrt ant. 



Kradiiig is rh»* -Mrdest thing 
I h<ive l»> dr, . 



Strongly 
LUsagree Disagree 



r 



T 



-2- 



Strongly Strongly 
Agree Agree Disagree Disagree 



8. I don't think a book is a 
very good birthday present, 



9. I often volunteer to reaci -H- -f 

dloucl ill schoDl, 



10. Reading, is often very boring. -H- 



]], I am a good reader. -t-f + 



12. I '>et worried when I ^^f'sked -H- 
to rt' ad sonieth itig . 



1"^. I like to read to people. 



I di>.i ih^ ^ u.i M- 



^ ERIC 



T 



Stronj;>l\ Strongly 
Agree Agree Disagree Disagree 



18^ I don't like to tell otl^er 
*'""v)ple about things I have 
re«id* 



I spend a l^t of rr^y time at 
home reading. 



20. I think I ai^ c^e of the best 
readers in my fciass. 



21. Mv classmates like to hear 
me reai. 



1^ 



I like to figure out new worfi^. , 



I (l^n't think I want to learn 
anot'hPr lan^^uage. 



- i r<*ao*> r . 



FRir 




T 



1 



Strongly 
ree 



Agree 



Stronf, }y 
Di s a^, ree i)i s aRroe 



, 1 never rend unless f^oineone 
forces ne . 



37. I read v^honever I have anv 
free lime.*. 



38* I arn a fast reader. 



30, I often start to read some- 
thing but give up because I 
don^t understand it. 



/^O. . .would like reading better 

s<j"ieon<» would help re with 

It , 



I Ilk.* ntart a ne\.' '5ook , 



^.'^i-n \ rtw^i 1/ t -Mr a for 



